-- Logs begin at Fri 2022-05-27 02:22:47 UTC, end at Fri 2022-05-27 03:41:56 UTC. -- May 27 03:08:07.526441 dv-cinder-828806-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 27 03:08:07.535430 dv-cinder-828806-4-iscsi devstack@keystone.service[80399]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 27 03:08:07.538395 dv-cinder-828806-4-iscsi devstack@keystone.service[80399]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri May 27 03:08:07 2022] *** May 27 03:08:07.538395 dv-cinder-828806-4-iscsi devstack@keystone.service[80399]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 27 03:08:07.538395 dv-cinder-828806-4-iscsi devstack@keystone.service[80399]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 27 03:08:07.538395 dv-cinder-828806-4-iscsi devstack@keystone.service[80399]: nodename: dv-cinder-828806-4-iscsi May 27 03:08:07.538395 dv-cinder-828806-4-iscsi devstack@keystone.service[80399]: machine: x86_64 May 27 03:08:07.538395 dv-cinder-828806-4-iscsi devstack@keystone.service[80399]: clock source: unix May 27 03:08:07.538395 dv-cinder-828806-4-iscsi devstack@keystone.service[80399]: pcre jit disabled May 27 03:08:07.538395 dv-cinder-828806-4-iscsi devstack@keystone.service[80399]: detected number of CPU cores: 4 May 27 03:08:07.538395 dv-cinder-828806-4-iscsi devstack@keystone.service[80399]: current working directory: / May 27 03:08:07.538395 dv-cinder-828806-4-iscsi devstack@keystone.service[80399]: detected binary path: /usr/bin/uwsgi-core May 27 03:08:07.538395 dv-cinder-828806-4-iscsi devstack@keystone.service[80399]: your processes number limit is 47680 May 27 03:08:07.539083 dv-cinder-828806-4-iscsi devstack@keystone.service[80399]: your memory page size is 4096 bytes May 27 03:08:07.539083 dv-cinder-828806-4-iscsi devstack@keystone.service[80399]: detected max file descriptor number: 2048 May 27 03:08:07.539083 dv-cinder-828806-4-iscsi devstack@keystone.service[80399]: lock engine: pthread robust mutexes May 27 03:08:07.539083 dv-cinder-828806-4-iscsi devstack@keystone.service[80399]: thunder lock: enabled May 27 03:08:07.539083 dv-cinder-828806-4-iscsi devstack@keystone.service[80399]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 27 03:08:07.539083 dv-cinder-828806-4-iscsi devstack@keystone.service[80399]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 27 03:08:07.583058 dv-cinder-828806-4-iscsi devstack@keystone.service[80399]: Python main interpreter initialized at 0x56253a61d9c0 May 27 03:08:07.583058 dv-cinder-828806-4-iscsi devstack@keystone.service[80399]: python threads support enabled May 27 03:08:07.583058 dv-cinder-828806-4-iscsi devstack@keystone.service[80399]: your server socket listen backlog is limited to 100 connections May 27 03:08:07.583058 dv-cinder-828806-4-iscsi devstack@keystone.service[80399]: your mercy for graceful operations on workers is 90 seconds May 27 03:08:07.583368 dv-cinder-828806-4-iscsi devstack@keystone.service[80399]: mapped 403077 bytes (393 KB) for 2 cores May 27 03:08:07.583368 dv-cinder-828806-4-iscsi devstack@keystone.service[80399]: *** Operational MODE: preforking *** May 27 03:08:07.583683 dv-cinder-828806-4-iscsi devstack@keystone.service[80399]: *** uWSGI is running in multiple interpreter mode *** May 27 03:08:07.583683 dv-cinder-828806-4-iscsi devstack@keystone.service[80399]: spawned uWSGI master process (pid: 80399) May 27 03:08:07.583898 dv-cinder-828806-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 27 03:08:07.584629 dv-cinder-828806-4-iscsi devstack@keystone.service[80399]: spawned uWSGI worker 1 (pid: 80409, cores: 1) May 27 03:08:07.584734 dv-cinder-828806-4-iscsi devstack@keystone.service[80399]: spawned uWSGI worker 2 (pid: 80410, cores: 1) May 27 03:08:07.586432 dv-cinder-828806-4-iscsi devstack@keystone.service[80399]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 27 03:08:13.277392 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 27 03:08:13.277392 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: from cryptography.utils import int_from_bytes May 27 03:08:13.278217 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 27 03:08:13.278217 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: from cryptography.utils import int_from_bytes May 27 03:08:13.327951 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 27 03:08:13.328140 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 27 03:08:13.328283 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] command line args: [] {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 27 03:08:13.328596 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 27 03:08:13.328729 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ================================================================================ {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 27 03:08:13.329053 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] admin_token = **** {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.329297 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] config_dir = [] {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.329450 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.329645 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] config_source = [] {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.329801 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] debug = True {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.330027 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.330356 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.330526 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.330809 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.331025 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.331256 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] list_limit = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.331460 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] log_config_append = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.331594 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.331767 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] log_dir = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.331945 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] log_file = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.332202 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.332438 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.332677 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.332920 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.333140 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.333299 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.333566 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.333789 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.334016 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.334246 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.334473 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.334707 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.334946 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.335186 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.335493 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.335971 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.50/identity {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.336150 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] publish_errors = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.336376 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.336602 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.336761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.337024 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.337259 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.337490 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] standard_threads = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.337716 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.337955 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.338128 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.338301 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] use_journal = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.338479 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] use_json = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.338651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] use_stderr = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.338823 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] use_syslog = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.339049 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.339300 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.339534 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.339789 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.340025 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.340269 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.340504 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.340747 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.340999 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.341247 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.341488 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.341735 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.341965 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.342207 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.342437 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.342681 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.342922 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.343156 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.343407 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.343657 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.343899 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.344131 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.344379 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.344901 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.344901 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.345164 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.345393 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.345620 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.345848 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.346079 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.346323 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.346567 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.346793 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.347020 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.347262 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.347507 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.347740 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.347984 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.348211 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.348447 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.348677 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.348901 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.349152 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.349378 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.349602 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.349843 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.350090 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.350318 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.350550 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.350775 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.351166 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.351410 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.351660 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.351885 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.352119 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.352352 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.352586 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.352818 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.353072 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.353302 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.353544 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.353776 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.354008 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.354234 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.354487 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.354723 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.354948 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.355180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.355411 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.355674 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.355911 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.356155 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.356398 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.356631 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.356864 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.357118 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.357364 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.357599 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.357832 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.358083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.358323 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.358557 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.358787 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.359020 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.359250 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.359507 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.359746 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.359984 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.360217 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.360455 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.360686 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.360919 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.361175 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.361401 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.361628 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.361854 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.362084 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.362310 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.362546 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.362777 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.363021 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.363252 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.363503 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.363756 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.363993 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.364218 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] auth.external = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.364444 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.364732 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.364986 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.365215 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] auth.password = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.365441 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] auth.token = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.365676 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.365906 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.366151 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.366378 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.366709 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.366954 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.367185 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.367514 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] credential.caching = True {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.367768 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.368018 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.368269 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.368503 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.368744 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.368994 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.369260 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.369512 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.369745 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.369996 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.370289 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.370534 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.370795 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.371036 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.371288 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.371520 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] federation.caching = True {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.371754 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.372000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.372244 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.372471 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.372714 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.372960 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.373219 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.373454 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.373704 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.373937 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.374183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.374415 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] identity.caching = True {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.374663 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.374907 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.375141 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.375377 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.375672 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.375899 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.376142 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.376391 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.376639 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.376867 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.377118 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.377347 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.377588 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.377831 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.378072 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.378328 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.378573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.378840 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.379074 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.379308 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.379534 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.379768 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.379996 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.380226 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.380463 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.380700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.380944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.381192 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.381435 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.381677 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.381906 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.382220 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.382551 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.382782 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.383019 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.383246 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.383481 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.383715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.383964 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.384198 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.384476 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.384784 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.385071 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.385302 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.385532 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.385779 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.386024 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.386259 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.386495 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.386727 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.386957 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.user = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.387193 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.387449 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.387677 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.387920 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.388168 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.388410 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.388640 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.388866 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.389122 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.389353 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.389586 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.389814 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.390056 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.390296 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.390536 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.390776 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.391019 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.391244 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.391486 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.391718 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.391952 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.392184 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.392414 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.392656 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.392899 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.393158 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.393409 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.393636 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.393880 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.394114 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.394346 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.394580 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.394658 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 27 03:08:13.394658 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: from cryptography.utils import int_from_bytes May 27 03:08:13.395160 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.395243 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 27 03:08:13.395243 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: from cryptography.utils import int_from_bytes May 27 03:08:13.395365 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.395365 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.395498 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.395706 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] resource.caching = True {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.395953 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.396198 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.396439 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.396686 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.396933 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.397189 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.397435 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.397669 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.397941 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.398177 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] role.caching = True {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.398421 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] role.driver = sql {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.398648 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.398894 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.399139 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.399383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.399626 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.399869 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.400215 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.400461 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.400706 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.400936 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.401204 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.401450 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.401701 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.401942 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.402344 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.402575 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.402821 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.403067 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.403312 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.403558 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.403791 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.404041 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.404288 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.404525 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.404755 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.405002 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.405236 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.405486 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.405739 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.405985 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.406220 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.406454 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.406681 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.406913 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] token.caching = True {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.407146 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.407392 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.407623 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.407882 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.408126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.408360 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.408601 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.408842 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.409110 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.409344 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.409579 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.409810 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.410051 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.410293 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.410518 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.410755 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.411046 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.411299 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] database.connection = **** {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.411851 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.412098 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.412332 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.412570 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.412802 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.413057 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.413291 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.413593 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.413834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.414068 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.414323 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.414557 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.414802 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.415036 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.415278 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.415505 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.415743 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.415975 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.416141 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 27 03:08:13.441927 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 27 03:08:13.442186 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 27 03:08:13.442267 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] command line args: [] {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 27 03:08:13.442566 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 27 03:08:13.442725 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ================================================================================ {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 27 03:08:13.443001 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] admin_token = **** {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.443226 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] config_dir = [] {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.443444 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.443584 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] config_source = [] {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.443762 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] debug = True {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.443998 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.444231 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.444366 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.444647 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.444921 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.445190 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] list_limit = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.445368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] log_config_append = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.445522 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.445917 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] log_dir = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.446063 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] log_file = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.446185 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.446422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.446683 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.446930 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.447107 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.447230 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.447357 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.447479 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.447602 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.447737 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.447878 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.448005 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.448186 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.448418 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.448648 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.448893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.449160 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.449435 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.449929 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.50/identity {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.450112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] publish_errors = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.450346 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.450579 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.450756 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.451003 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.451237 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.451469 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] standard_threads = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.451698 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.451952 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.452126 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.452301 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] use_journal = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.452475 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] use_json = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.452647 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] use_stderr = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.452821 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] use_syslog = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.453075 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 03:08:13.453338 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.453581 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.453843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.454083 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.454334 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.454573 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.454821 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.455056 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.455304 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.455551 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.455802 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.456035 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.456248 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.456377 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.456510 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.456735 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.456998 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.457237 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.457490 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.457745 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.457989 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.458223 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.458475 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.458736 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.458989 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.459237 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.459469 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.459698 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.459929 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.460161 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.461293 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.461599 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.461840 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.462074 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.462323 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.462572 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.462812 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.463060 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.463293 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.463528 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.463762 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.463993 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.464223 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.464455 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.464684 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.464920 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.465192 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.465423 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.465660 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.465889 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.466125 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.466358 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.466605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.466839 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.467074 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.467309 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.467544 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.467779 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.468022 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.468253 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.468502 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.468737 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.468994 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.469224 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.469485 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.469721 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.469951 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.470187 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.470421 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.470689 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.470927 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.471178 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.471421 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.471657 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.471894 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.472127 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.472373 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.472611 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.472846 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.473108 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.473344 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.473578 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.473812 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.474048 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.474281 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.474541 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.474781 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.475021 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.475257 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.475498 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.475748 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.476058 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.476293 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.476612 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.476849 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.477103 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.477341 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.477573 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.477814 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.478048 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.478297 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.478595 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.478791 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.479063 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.479304 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.479537 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] auth.external = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.479767 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.480063 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.480303 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.480536 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] auth.password = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.480768 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] auth.token = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.481035 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.481274 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.481525 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.481756 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.482004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.482252 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.482486 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.482722 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] credential.caching = True {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.482971 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.483278 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.483619 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.483861 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.484109 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.484347 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.484610 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.484694 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.484855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.484931 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.485114 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.485370 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.485659 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.485906 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.486170 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.486414 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.486669 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.486904 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] federation.caching = True {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.487141 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.487387 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.487634 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.487862 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.488109 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.488346 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.488600 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.488843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.489191 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.489430 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.489686 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.489913 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] identity.caching = True {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.490161 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.490406 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.490643 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.490878 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.491130 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.491358 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.491690 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.491950 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.492203 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.492438 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.492673 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.492974 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.493217 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.493558 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.493906 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.494261 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.494608 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.494980 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.495276 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.495612 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.495943 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.496268 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.496567 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.496811 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.497079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.497325 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.497576 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.497810 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.498060 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.498310 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.498547 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.498794 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.499042 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.499274 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.499513 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.499743 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.499983 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.500219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.500464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.500701 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.500944 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.501214 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.501463 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.501693 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.501926 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.502175 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.502422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.502655 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.502890 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.503125 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.503354 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.user = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.503592 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.503975 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.504101 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.504324 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.504571 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.504816 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.505078 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.505311 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.505546 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.505781 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.506017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.506247 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.506494 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.506740 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.506985 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.507315 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.507630 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.507776 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.508017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.508234 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.508524 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.508707 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.508935 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.509691 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.509956 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.510200 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.510462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.510699 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.510948 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.511197 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.511437 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.511676 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.511924 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.512168 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.512401 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.512637 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.512876 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] resource.caching = True {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.513156 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.513988 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.513988 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.513988 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.514227 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.514365 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.514615 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.514854 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.515095 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.515329 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] role.caching = True {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.515577 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] role.driver = sql {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.515808 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.516058 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.516306 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.516554 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.516801 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.517095 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.517343 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.517589 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.517837 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.518070 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.518316 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.518562 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.518807 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.519053 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.519473 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.519708 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.519959 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.520207 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.520453 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.520701 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.520937 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.521333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.521584 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.521823 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.522056 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.522289 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.522520 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.522786 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.523047 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.523298 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.523537 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.523774 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.524006 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.524242 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] token.caching = True {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.524479 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.524742 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.524984 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.525237 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.525485 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.526067 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.526067 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.526268 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.526438 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.526676 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.526914 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.527148 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.527439 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.527643 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.527874 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.528117 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.528417 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.528677 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] database.connection = **** {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.528922 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.529190 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.529432 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.529675 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.529911 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.530148 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.530385 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.530620 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.530864 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.531107 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.531351 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.531586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.531832 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.532069 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.532322 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.532561 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.532810 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.533072 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 03:08:13.533249 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80409) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 27 03:08:13.538077 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.558958 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.559193 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.559414 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.569441 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.588250 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.588483 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.589860 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.590093 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.626051 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.655325 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.655570 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.655774 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.655990 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.656212 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.656432 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.663851 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.664099 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.680210 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.680424 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.680667 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.680894 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.681139 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.681357 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.681569 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.693073 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.705606 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.705843 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.706068 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.706285 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.706504 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.706696 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.706930 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80410) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.707792 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.708031 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.708253 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.708476 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.708693 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.708893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.722081 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.722315 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.722543 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.722763 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.722983 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.723198 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.723410 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.726037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.726446 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 27 03:08:13.727188 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.727445 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 27 03:08:13.727722 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.727985 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.728223 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.728471 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.731810 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.734948 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.735196 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.735422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.735651 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.735849 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.736086 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.736293 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80409) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 03:08:13.739479 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.739764 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.740026 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.740299 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.747020 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.747224 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 27 03:08:13.747574 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.747770 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 27 03:08:13.748048 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.748316 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.748457 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:08:13.748575 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.748840 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.751741 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:08:13.752019 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.752301 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.752582 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.752861 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.753145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.753522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.753799 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.754064 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.754334 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.754602 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.759940 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.760217 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.760490 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.762523 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.770735 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80409) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:08:13.774220 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80409) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:08:13.774496 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.774778 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.775066 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.775351 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.775617 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.775877 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.776164 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.776428 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.776696 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.777078 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.781024 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.782737 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:08:13.783014 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.789137 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:08:13.789137 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.789403 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.796482 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:08:13.796768 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.801285 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:08:13.801665 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.801948 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.802224 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.802498 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.802774 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.803032 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.803302 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.805127 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.806789 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80409) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:08:13.807094 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.812561 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80409) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:08:13.812842 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.813144 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.820508 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80409) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:08:13.820794 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.822087 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.822363 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.825335 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80409) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:08:13.825619 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.825900 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.826179 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.826464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.826740 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.827004 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:08:13.827100 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.827347 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.830656 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.830946 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.836068 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:08:13.839650 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:08:13.843422 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.843719 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.844009 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.844293 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.844572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.844842 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.846005 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.846284 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.850982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80409) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:08:13.854699 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.854994 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.859955 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80409) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:08:13.863637 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80409) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:08:13.863761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:08:13.863890 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.864165 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.864428 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.867361 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.867657 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.867940 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.868221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.868503 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.868903 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.872423 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.874243 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.874530 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.877592 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:08:13.877871 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.878148 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.878424 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.878691 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.887746 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80409) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:08:13.888010 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.888299 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.888555 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.892357 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:08:13.892637 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.892916 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.893232 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.893508 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.893782 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.894064 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.896552 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.898345 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.898617 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.901667 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80409) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:08:13.901949 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.902233 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.902518 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.902797 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.912408 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:08:13.915715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:08:13.916697 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80409) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:08:13.916991 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.917281 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.917558 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.917841 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.918123 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.918405 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.919040 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.920629 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.922292 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:08:13.922579 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.922872 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.930300 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.931936 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:08:13.935300 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.935589 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.935863 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.936131 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.936299 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80409) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:08:13.939877 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80409) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:08:13.943069 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.944731 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.945197 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:08:13.945504 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.945878 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.946475 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80409) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:08:13.946775 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.947064 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.953702 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80410) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:08:13.953975 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.954223 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.954352 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.954512 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.954773 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.955041 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.955346 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.955578 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.955827 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.956063 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80409) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:08:13.956148 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.956378 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.956629 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.956902 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.957169 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80410) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.959254 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.959560 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.959828 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.960099 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.969317 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80409) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:08:13.969600 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.969875 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.978046 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80409) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 03:08:13.978322 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.978595 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.978868 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.979138 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.979437 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.979687 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.979955 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.980211 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.980485 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.980764 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.981049 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.981322 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.981579 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80409) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 03:08:13.995957 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56253a61d9c0 pid: 80410 (default app) May 27 03:08:13.999274 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326729f5-5ebd-41d1-87ae-30e8de7d3ef5 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:13.999566 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326729f5-5ebd-41d1-87ae-30e8de7d3ef5 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:13.999840 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326729f5-5ebd-41d1-87ae-30e8de7d3ef5 None None] PATH_INFO: `/v3` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:14.004186 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1/1] 10.222.0.50 () {50 vars in 791 bytes} [Fri May 27 03:08:13 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:14.020579 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x56253a61d9c0 pid: 80409 (default app) May 27 03:08:22.025763 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007d160-6899-4f80-990e-772b1445fc24 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:22.026361 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007d160-6899-4f80-990e-772b1445fc24 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:22.026361 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007d160-6899-4f80-990e-772b1445fc24 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:22.027664 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 2/2] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:08:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:08:22.035080 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03104c0-9a6d-422c-8a05-4bf6778f5492 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:22.035521 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03104c0-9a6d-422c-8a05-4bf6778f5492 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:22.035610 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03104c0-9a6d-422c-8a05-4bf6778f5492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:22.102811 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: INFO dbcounter [None req-e03104c0-9a6d-422c-8a05-4bf6778f5492 None None] Registered counter for database keystone May 27 03:08:22.128003 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG oslo_db.sqlalchemy.engines [None req-e03104c0-9a6d-422c-8a05-4bf6778f5492 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80409) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 27 03:08:22.141256 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writer thread running {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 27 03:08:22.323637 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG passlib.handlers.bcrypt [None req-e03104c0-9a6d-422c-8a05-4bf6778f5492 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80409) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 27 03:08:22.324055 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG passlib.handlers.bcrypt [None req-e03104c0-9a6d-422c-8a05-4bf6778f5492 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80409) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 27 03:08:22.381798 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-e03104c0-9a6d-422c-8a05-4bf6778f5492 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:08:22.383453 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-e03104c0-9a6d-422c-8a05-4bf6778f5492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:08:22.518527 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1/3] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:08:22 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 487 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 27 03:08:22.761732 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2007d160-6899-4f80-990e-772b1445fc24 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:22.835893 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: INFO dbcounter [None req-2007d160-6899-4f80-990e-772b1445fc24 None None] Registered counter for database keystone May 27 03:08:22.857594 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG oslo_db.sqlalchemy.engines [None req-2007d160-6899-4f80-990e-772b1445fc24 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80410) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 27 03:08:22.868972 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writer thread running {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 27 03:08:22.989787 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2007d160-6899-4f80-990e-772b1445fc24 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:22.995312 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d84705e-66fd-4b9c-b5b0-6481211d6eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:22.995980 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d84705e-66fd-4b9c-b5b0-6481211d6eca admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:22.996257 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d84705e-66fd-4b9c-b5b0-6481211d6eca admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:22.996533 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d84705e-66fd-4b9c-b5b0-6481211d6eca admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:23.006160 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d84705e-66fd-4b9c-b5b0-6481211d6eca admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:23.057956 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2d84705e-66fd-4b9c-b5b0-6481211d6eca admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 27 03:08:23.058525 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d84705e-66fd-4b9c-b5b0-6481211d6eca admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:23.068434 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 3/4] 10.222.0.50 () {58 vars in 1324 bytes} [Fri May 27 03:08:22 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:08:23.075229 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03104c0-9a6d-422c-8a05-4bf6778f5492 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:23.090677 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03104c0-9a6d-422c-8a05-4bf6778f5492 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:23.096268 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5891c22-2f88-4bdd-adb7-99ddc0ff5f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:23.097439 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5891c22-2f88-4bdd-adb7-99ddc0ff5f35 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:23.097720 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5891c22-2f88-4bdd-adb7-99ddc0ff5f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:23.097982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5891c22-2f88-4bdd-adb7-99ddc0ff5f35 admin admin] PATH_INFO: `/v3/projects/73644351894d46fdb8889fca96b4c614` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:23.099743 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5891c22-2f88-4bdd-adb7-99ddc0ff5f35 admin admin] RBAC: Authorizing `identity:get_project(project_id=73644351894d46fdb8889fca96b4c614)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:23.152245 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a5891c22-2f88-4bdd-adb7-99ddc0ff5f35 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 27 03:08:23.152577 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5891c22-2f88-4bdd-adb7-99ddc0ff5f35 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:23.157215 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 2/5] 10.222.0.50 () {56 vars in 1207 bytes} [Fri May 27 03:08:23 2022] GET /identity/v3/projects/73644351894d46fdb8889fca96b4c614 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:08:25.324164 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b2487-ab20-45de-84cd-332551f1e0a5 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:25.324633 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b2487-ab20-45de-84cd-332551f1e0a5 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:25.324749 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b2487-ab20-45de-84cd-332551f1e0a5 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:25.328860 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 4/6] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:08:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:08:25.334663 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea90563-8acd-4e23-af42-c8423852893c None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:25.335090 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea90563-8acd-4e23-af42-c8423852893c None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:25.335176 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea90563-8acd-4e23-af42-c8423852893c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:25.357121 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-cea90563-8acd-4e23-af42-c8423852893c None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:08:25.358657 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-cea90563-8acd-4e23-af42-c8423852893c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:08:25.367613 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 3/7] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:08:25 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 27 03:08:25.390099 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b2487-ab20-45de-84cd-332551f1e0a5 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:25.405712 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b2487-ab20-45de-84cd-332551f1e0a5 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:25.411612 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cde192-9e43-4706-a0b5-038cde37cf03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:25.412287 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cde192-9e43-4706-a0b5-038cde37cf03 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:25.412569 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cde192-9e43-4706-a0b5-038cde37cf03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:25.412842 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cde192-9e43-4706-a0b5-038cde37cf03 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:25.421230 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cde192-9e43-4706-a0b5-038cde37cf03 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:25.423064 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cde192-9e43-4706-a0b5-038cde37cf03 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:25.432731 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 5/8] 10.222.0.50 () {58 vars in 1324 bytes} [Fri May 27 03:08:25 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:08:25.438615 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea90563-8acd-4e23-af42-c8423852893c None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:25.448113 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea90563-8acd-4e23-af42-c8423852893c None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:25.454629 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fc05dd-e137-4772-a45e-ae979ff9e7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:25.455510 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc05dd-e137-4772-a45e-ae979ff9e7aa admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:25.455790 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc05dd-e137-4772-a45e-ae979ff9e7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:25.456047 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fc05dd-e137-4772-a45e-ae979ff9e7aa admin admin] PATH_INFO: `/v3/users/4a96d7acf93e46c7ac5700560db36dbe` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:25.457671 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fc05dd-e137-4772-a45e-ae979ff9e7aa admin admin] RBAC: Authorizing `identity:get_user(user_id=4a96d7acf93e46c7ac5700560db36dbe)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:25.459549 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fc05dd-e137-4772-a45e-ae979ff9e7aa admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:25.461225 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 4/9] 10.222.0.50 () {56 vars in 1198 bytes} [Fri May 27 03:08:25 2022] GET /identity/v3/users/4a96d7acf93e46c7ac5700560db36dbe => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:27.720541 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f1778-ded7-47d6-a385-d7910702e14a None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:27.721103 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f1778-ded7-47d6-a385-d7910702e14a None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:27.721192 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f1778-ded7-47d6-a385-d7910702e14a None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:27.722503 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 6/10] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:08:27.728544 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74ac93-01ff-4336-a243-b1ada57b0711 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:27.729113 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74ac93-01ff-4336-a243-b1ada57b0711 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:27.729200 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f856ba-4cd1-4756-a829-27de0e3c7110 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:27.729284 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74ac93-01ff-4336-a243-b1ada57b0711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:27.729374 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f856ba-4cd1-4756-a829-27de0e3c7110 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:27.729549 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f856ba-4cd1-4756-a829-27de0e3c7110 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:27.730870 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 7/11] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:08:27.736551 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d6398-0cac-40ab-94f2-aeb711beb778 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:27.736838 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d6398-0cac-40ab-94f2-aeb711beb778 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:27.737160 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d6398-0cac-40ab-94f2-aeb711beb778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:27.753698 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-5b74ac93-01ff-4336-a243-b1ada57b0711 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:08:27.755533 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-5b74ac93-01ff-4336-a243-b1ada57b0711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:08:27.784753 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG passlib.handlers.bcrypt [None req-b43d6398-0cac-40ab-94f2-aeb711beb778 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80410) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 27 03:08:27.785218 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG passlib.handlers.bcrypt [None req-b43d6398-0cac-40ab-94f2-aeb711beb778 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80410) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 27 03:08:27.831799 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-b43d6398-0cac-40ab-94f2-aeb711beb778 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:08:27.833436 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-b43d6398-0cac-40ab-94f2-aeb711beb778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:08:27.840032 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 5/12] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:08:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:08:27.863738 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b74ac93-01ff-4336-a243-b1ada57b0711 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:27.888523 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:27.935479 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 8/13] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:08:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:08:27.957679 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d6398-0cac-40ab-94f2-aeb711beb778 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:27.970650 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b74ac93-01ff-4336-a243-b1ada57b0711 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:28.008930 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406938b5-107d-47b6-9b3f-5743f04592b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:28.009766 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406938b5-107d-47b6-9b3f-5743f04592b8 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:28.010051 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406938b5-107d-47b6-9b3f-5743f04592b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:28.010324 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406938b5-107d-47b6-9b3f-5743f04592b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:28.023703 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406938b5-107d-47b6-9b3f-5743f04592b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:08:28.035317 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-406938b5-107d-47b6-9b3f-5743f04592b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:08:28.035317 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:08:28.035317 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:08:28.035317 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:08:28.035317 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:08:28.035317 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:08:28.035317 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:08:28.035317 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:08:28.035317 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:08:28.035317 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:08:28.035317 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:08:28.035317 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:08:28.035317 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:08:28.035317 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:08:28.035317 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:08:28.035317 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:08:28.036853 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:08:28.036853 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:08:28.036853 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:08:28.036853 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:08:28.036853 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:08:28.036853 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:08:28.036853 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:08:28.036853 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:08:28.036853 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:08:28.036853 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:08:28.036853 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:08:28.036853 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:08:28.036853 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:08:28.036853 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:08:28.036853 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:08:28.036853 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:08:28.037760 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:08:28.037760 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:08:28.037760 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:08:28.037760 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:08:28.037760 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:08:28.037760 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:08:28.037760 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:08:28.037760 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:08:28.037760 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:08:28.037760 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:08:28.037760 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:08:28.037760 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:08:28.037760 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 6/14] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 03:08:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:08:28.043436 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406938b5-107d-47b6-9b3f-5743f04592b8 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:28.182991 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d6398-0cac-40ab-94f2-aeb711beb778 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:28.185662 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:28.224666 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4dd06-df54-4a92-882c-3940f9df484a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:28.225533 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4dd06-df54-4a92-882c-3940f9df484a None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:28.225821 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4dd06-df54-4a92-882c-3940f9df484a None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:28.226100 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4dd06-df54-4a92-882c-3940f9df484a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:28.248759 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:28.270117 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406938b5-107d-47b6-9b3f-5743f04592b8 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:28.300640 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c4dd06-df54-4a92-882c-3940f9df484a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:28.302222 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c4dd06-df54-4a92-882c-3940f9df484a None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:28.307674 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62373083-6b36-4d84-b357-d7cbfdc51bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:28.308351 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62373083-6b36-4d84-b357-d7cbfdc51bea None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:28.308629 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62373083-6b36-4d84-b357-d7cbfdc51bea None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:28.308935 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62373083-6b36-4d84-b357-d7cbfdc51bea None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:28.309721 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62373083-6b36-4d84-b357-d7cbfdc51bea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:08:28.310028 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62373083-6b36-4d84-b357-d7cbfdc51bea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:28.311429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62373083-6b36-4d84-b357-d7cbfdc51bea None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:28.323645 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 7/15] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:08:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:08:28.329286 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62373083-6b36-4d84-b357-d7cbfdc51bea None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:28.402382 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 9/16] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:08:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:08:28.408150 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4dd06-df54-4a92-882c-3940f9df484a None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:28.411944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:28.430266 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62373083-6b36-4d84-b357-d7cbfdc51bea None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:28.463410 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253b971a-2625-4698-a0f7-dd79ff8f74b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:28.464295 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253b971a-2625-4698-a0f7-dd79ff8f74b7 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:28.464569 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253b971a-2625-4698-a0f7-dd79ff8f74b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:28.464839 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253b971a-2625-4698-a0f7-dd79ff8f74b7 None admin] PATH_INFO: `/v3/users/4a96d7acf93e46c7ac5700560db36dbe` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:28.466058 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253b971a-2625-4698-a0f7-dd79ff8f74b7 None admin] RBAC: Authorizing `identity:get_user(user_id=4a96d7acf93e46c7ac5700560db36dbe)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:28.467557 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253b971a-2625-4698-a0f7-dd79ff8f74b7 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:28.469430 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 8/17] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:08:28 2022] GET /identity/v3/users/4a96d7acf93e46c7ac5700560db36dbe => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:08:28.474635 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253b971a-2625-4698-a0f7-dd79ff8f74b7 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:28.489394 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:28.517203 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4dd06-df54-4a92-882c-3940f9df484a None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:28.550301 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a517d454-c8ef-472f-8464-ea80ef4460a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:28.551156 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517d454-c8ef-472f-8464-ea80ef4460a1 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:28.551438 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517d454-c8ef-472f-8464-ea80ef4460a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:28.551736 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517d454-c8ef-472f-8464-ea80ef4460a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:28.572449 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a517d454-c8ef-472f-8464-ea80ef4460a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:28.573988 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a517d454-c8ef-472f-8464-ea80ef4460a1 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:28.578169 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253b971a-2625-4698-a0f7-dd79ff8f74b7 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:28.595496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-a517d454-c8ef-472f-8464-ea80ef4460a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:08:28.595496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:08:28.595496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:08:28.595496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:08:28.595496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:08:28.595496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:08:28.595496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:08:28.595496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:08:28.595496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:08:28.595496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:08:28.595496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:08:28.595496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:08:28.595496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:08:28.595496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:08:28.595496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:08:28.596477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:08:28.596477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:08:28.596477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:08:28.596477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:08:28.596477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:08:28.596477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:08:28.596477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:08:28.596477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:08:28.596477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:08:28.596477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:08:28.596477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:08:28.596477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:08:28.596477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:08:28.596477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:08:28.596477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:08:28.596477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:08:28.596477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:08:28.602475 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:08:28.602475 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:08:28.602475 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:08:28.602475 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:08:28.602475 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 10/18] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:08:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:08:28.603030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a517d454-c8ef-472f-8464-ea80ef4460a1 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:28.612259 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13c2631-e03a-4444-a550-71c0f132bf4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:28.613101 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c2631-e03a-4444-a550-71c0f132bf4e None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:28.613380 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c2631-e03a-4444-a550-71c0f132bf4e None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:28.613647 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c2631-e03a-4444-a550-71c0f132bf4e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:28.614824 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13c2631-e03a-4444-a550-71c0f132bf4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:28.616294 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13c2631-e03a-4444-a550-71c0f132bf4e None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:28.618261 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 9/19] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:08:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:08:28.623896 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13c2631-e03a-4444-a550-71c0f132bf4e None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:28.662978 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:28.709040 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a517d454-c8ef-472f-8464-ea80ef4460a1 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:28.710597 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:28.731322 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13c2631-e03a-4444-a550-71c0f132bf4e None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:28.745054 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa688e-b16d-45aa-b4bd-5342216a8baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:28.745667 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa688e-b16d-45aa-b4bd-5342216a8baf None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:28.745954 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa688e-b16d-45aa-b4bd-5342216a8baf None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:28.746227 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa688e-b16d-45aa-b4bd-5342216a8baf None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:28.746979 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaa688e-b16d-45aa-b4bd-5342216a8baf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:08:28.747288 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaa688e-b16d-45aa-b4bd-5342216a8baf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:28.748684 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaa688e-b16d-45aa-b4bd-5342216a8baf None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:28.766004 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 11/20] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:08:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:28.766416 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa5040-4d49-475b-bdfd-6f105c7e4d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:28.767081 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa5040-4d49-475b-bdfd-6f105c7e4d16 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:28.767364 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa5040-4d49-475b-bdfd-6f105c7e4d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:28.767640 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa5040-4d49-475b-bdfd-6f105c7e4d16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:28.768478 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aa5040-4d49-475b-bdfd-6f105c7e4d16 None admin] RBAC: Adding query filter params (role.id=b4c524dd7bb44e94bd3b42d86e1cdf42, scope.domain.id=default, user.id=4a96d7acf93e46c7ac5700560db36dbe) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:08:28.768789 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aa5040-4d49-475b-bdfd-6f105c7e4d16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:28.770307 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aa5040-4d49-475b-bdfd-6f105c7e4d16 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:28.778394 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 10/21] 10.222.0.50 () {56 vars in 1322 bytes} [Fri May 27 03:08:28 2022] GET /identity/v3/role_assignments?user.id=4a96d7acf93e46c7ac5700560db36dbe&scope.domain.id=default&role.id=b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 233 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:08:31.563041 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f92a07-2eac-4058-9e75-8fa2df91ef26 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:31.564036 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f92a07-2eac-4058-9e75-8fa2df91ef26 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:31.564522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f92a07-2eac-4058-9e75-8fa2df91ef26 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:31.566321 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 12/22] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:08:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:08:31.582605 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a127e-e50d-48f1-a93c-ffe927fcc144 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:31.582808 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1708d-06d9-4273-852d-6918a4655cd1 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:31.583212 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1708d-06d9-4273-852d-6918a4655cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:31.583212 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a1708d-06d9-4273-852d-6918a4655cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:31.583798 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a127e-e50d-48f1-a93c-ffe927fcc144 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:31.584298 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a127e-e50d-48f1-a93c-ffe927fcc144 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:31.586165 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 13/23] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:08:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:08:31.592003 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f6ee7-908f-4ea4-ad51-b52c8eb91fb4 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:31.592526 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f6ee7-908f-4ea4-ad51-b52c8eb91fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:31.593053 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f6ee7-908f-4ea4-ad51-b52c8eb91fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:31.609972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-95a1708d-06d9-4273-852d-6918a4655cd1 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:08:31.611498 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-95a1708d-06d9-4273-852d-6918a4655cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:08:31.626220 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-fd1f6ee7-908f-4ea4-ad51-b52c8eb91fb4 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:08:31.627917 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-fd1f6ee7-908f-4ea4-ad51-b52c8eb91fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:08:31.735270 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 11/24] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:08:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:08:31.736636 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eccf51-f88a-4d11-b6d2-f1243203b048 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:31.736925 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eccf51-f88a-4d11-b6d2-f1243203b048 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:31.737228 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eccf51-f88a-4d11-b6d2-f1243203b048 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:31.740011 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 12/25] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:08:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:08:31.750468 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f810d6a-3623-4dd2-b3c0-4b6854d7a553 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:31.750758 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f810d6a-3623-4dd2-b3c0-4b6854d7a553 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:31.751040 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f810d6a-3623-4dd2-b3c0-4b6854d7a553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:31.773575 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 14/26] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:08:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:08:31.774527 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f6ee7-908f-4ea4-ad51-b52c8eb91fb4 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:31.782061 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-5f810d6a-3623-4dd2-b3c0-4b6854d7a553 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:08:31.783282 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-5f810d6a-3623-4dd2-b3c0-4b6854d7a553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:08:31.876123 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:31.911859 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 13/27] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:08:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:08:31.912811 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f810d6a-3623-4dd2-b3c0-4b6854d7a553 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:31.921945 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:31.987080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f6ee7-908f-4ea4-ad51-b52c8eb91fb4 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:32.025325 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ebdc1a-020a-41bb-a33f-da5f02abf290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:32.026503 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ebdc1a-020a-41bb-a33f-da5f02abf290 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:32.027014 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ebdc1a-020a-41bb-a33f-da5f02abf290 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:32.027500 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ebdc1a-020a-41bb-a33f-da5f02abf290 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:32.042868 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ebdc1a-020a-41bb-a33f-da5f02abf290 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:08:32.054344 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-a0ebdc1a-020a-41bb-a33f-da5f02abf290 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:08:32.054344 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:08:32.054344 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:08:32.054344 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:08:32.054344 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:08:32.054344 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:08:32.054344 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:08:32.054344 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:08:32.054344 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:08:32.054344 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:08:32.054344 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:08:32.054344 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:08:32.054344 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:08:32.054344 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:08:32.054344 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:08:32.054344 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:08:32.055480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:08:32.055480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:08:32.055480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:08:32.055480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:08:32.055480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:08:32.055480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:08:32.055480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:08:32.055480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:08:32.055480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:08:32.055480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:08:32.055480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:08:32.055480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:08:32.055480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:08:32.055480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:08:32.055480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:08:32.055480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:08:32.056222 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:08:32.056222 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:08:32.056222 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:08:32.056222 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:08:32.056222 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:08:32.056222 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:08:32.056222 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:08:32.056222 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:08:32.056222 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:08:32.056222 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:08:32.056222 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:08:32.056222 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:08:32.059205 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 15/28] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 03:08:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:08:32.060333 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ebdc1a-020a-41bb-a33f-da5f02abf290 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:32.163356 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f810d6a-3623-4dd2-b3c0-4b6854d7a553 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:32.187407 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ebdc1a-020a-41bb-a33f-da5f02abf290 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:32.195695 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:32.224655 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9037f119-98ed-4092-bac6-e1887b124a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:32.225373 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037f119-98ed-4092-bac6-e1887b124a6f None admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:32.225655 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037f119-98ed-4092-bac6-e1887b124a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:32.225989 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9037f119-98ed-4092-bac6-e1887b124a6f None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:32.226857 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9037f119-98ed-4092-bac6-e1887b124a6f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:32.228476 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9037f119-98ed-4092-bac6-e1887b124a6f None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:32.270583 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe4b040-2f13-4874-80cb-81e7a2030448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:32.271791 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4b040-2f13-4874-80cb-81e7a2030448 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:32.272302 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4b040-2f13-4874-80cb-81e7a2030448 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:32.273036 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4b040-2f13-4874-80cb-81e7a2030448 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:32.285351 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe4b040-2f13-4874-80cb-81e7a2030448 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:32.287580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe4b040-2f13-4874-80cb-81e7a2030448 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:32.308612 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-efe4b040-2f13-4874-80cb-81e7a2030448 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:08:32.308612 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:08:32.308612 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:08:32.308612 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:08:32.308612 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:08:32.308612 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:08:32.308612 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:08:32.308612 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:08:32.308612 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:08:32.308612 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:08:32.308612 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:08:32.308612 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:08:32.308612 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:08:32.308612 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:08:32.308612 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:08:32.309630 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:08:32.309630 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:08:32.309630 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:08:32.309630 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:08:32.309630 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:08:32.309630 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:08:32.309630 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:08:32.309630 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:08:32.309630 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:08:32.309630 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:08:32.309630 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:08:32.309630 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:08:32.309630 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:08:32.309630 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:08:32.309630 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:08:32.309630 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:08:32.309630 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:08:32.310431 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:08:32.310431 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:08:32.310431 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:08:32.310431 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:08:32.312337 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 14/29] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:08:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:08:32.313557 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe4b040-2f13-4874-80cb-81e7a2030448 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:32.375509 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:32.390444 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 16/30] 10.222.0.50 () {60 vars in 1131 bytes} [Fri May 27 03:08:32 2022] POST /identity/v3/roles => generated 219 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:08:32.392031 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61146268-7c8c-4ca1-a297-16ebfc4477b8 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:32.392332 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61146268-7c8c-4ca1-a297-16ebfc4477b8 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:32.392616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61146268-7c8c-4ca1-a297-16ebfc4477b8 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:32.393919 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 17/31] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:08:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:08:32.394668 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61146268-7c8c-4ca1-a297-16ebfc4477b8 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:32.439701 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe4b040-2f13-4874-80cb-81e7a2030448 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:32.486230 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afbe5bd-9b8d-4a64-9ba3-427458f17019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:32.487340 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afbe5bd-9b8d-4a64-9ba3-427458f17019 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:32.487816 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afbe5bd-9b8d-4a64-9ba3-427458f17019 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:32.488279 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afbe5bd-9b8d-4a64-9ba3-427458f17019 None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:32.489996 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afbe5bd-9b8d-4a64-9ba3-427458f17019 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:08:32.490573 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afbe5bd-9b8d-4a64-9ba3-427458f17019 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:32.493108 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afbe5bd-9b8d-4a64-9ba3-427458f17019 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:32.505425 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 15/32] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:08:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:32.507605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5340d-04c6-43b0-ad46-daad9261c37f None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:32.508267 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5340d-04c6-43b0-ad46-daad9261c37f None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:32.508733 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5340d-04c6-43b0-ad46-daad9261c37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:32.520134 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61146268-7c8c-4ca1-a297-16ebfc4477b8 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:32.532702 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-6ad5340d-04c6-43b0-ad46-daad9261c37f None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:08:32.534836 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-6ad5340d-04c6-43b0-ad46-daad9261c37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:08:32.562926 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44560f4b-e1d5-4b04-b7d5-cc35110ed447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:32.563815 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44560f4b-e1d5-4b04-b7d5-cc35110ed447 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:32.564368 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44560f4b-e1d5-4b04-b7d5-cc35110ed447 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:32.564526 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44560f4b-e1d5-4b04-b7d5-cc35110ed447 None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:32.565701 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44560f4b-e1d5-4b04-b7d5-cc35110ed447 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:08:32.565908 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44560f4b-e1d5-4b04-b7d5-cc35110ed447 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:32.567649 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44560f4b-e1d5-4b04-b7d5-cc35110ed447 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:32.581464 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 18/33] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:08:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:32.583168 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723cc20-4fb3-4d93-b98a-885612d497e3 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:32.583571 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723cc20-4fb3-4d93-b98a-885612d497e3 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:32.583944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723cc20-4fb3-4d93-b98a-885612d497e3 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:32.585967 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 19/34] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:08:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:08:32.587303 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0723cc20-4fb3-4d93-b98a-885612d497e3 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:32.619309 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:32.634569 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 16/35] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:08:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:08:32.636724 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f1fe1c-20c8-44e6-9cb3-e0fa16e1ad52 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:32.637263 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f1fe1c-20c8-44e6-9cb3-e0fa16e1ad52 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:32.637732 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f1fe1c-20c8-44e6-9cb3-e0fa16e1ad52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:32.656023 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-00f1fe1c-20c8-44e6-9cb3-e0fa16e1ad52 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:08:32.657739 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-00f1fe1c-20c8-44e6-9cb3-e0fa16e1ad52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:08:32.707889 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:32.728944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0723cc20-4fb3-4d93-b98a-885612d497e3 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:32.743260 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 17/36] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:08:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:08:32.743260 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f1fe1c-20c8-44e6-9cb3-e0fa16e1ad52 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:32.764083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8907f5e-060c-4cf4-82bd-c7844cc402d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:32.765011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8907f5e-060c-4cf4-82bd-c7844cc402d3 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:32.765292 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8907f5e-060c-4cf4-82bd-c7844cc402d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:32.765571 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8907f5e-060c-4cf4-82bd-c7844cc402d3 None admin] PATH_INFO: `/v3/users/4a96d7acf93e46c7ac5700560db36dbe` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:32.767325 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8907f5e-060c-4cf4-82bd-c7844cc402d3 None admin] RBAC: Authorizing `identity:get_user(user_id=4a96d7acf93e46c7ac5700560db36dbe)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:32.768832 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8907f5e-060c-4cf4-82bd-c7844cc402d3 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:32.774991 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 20/37] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:08:32 2022] GET /identity/v3/users/4a96d7acf93e46c7ac5700560db36dbe => generated 247 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:32.774991 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8907f5e-060c-4cf4-82bd-c7844cc402d3 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:32.873299 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f1fe1c-20c8-44e6-9cb3-e0fa16e1ad52 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:32.878796 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:32.887775 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8907f5e-060c-4cf4-82bd-c7844cc402d3 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:32.909996 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ffc74-851a-4a0f-a76f-9f644d13bc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:32.910668 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ffc74-851a-4a0f-a76f-9f644d13bc69 None admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:32.910951 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ffc74-851a-4a0f-a76f-9f644d13bc69 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:32.911583 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ffc74-851a-4a0f-a76f-9f644d13bc69 None admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:32.912295 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ffc74-851a-4a0f-a76f-9f644d13bc69 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:32.913987 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ffc74-851a-4a0f-a76f-9f644d13bc69 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:32.925446 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a84de-e19e-406e-b188-1a2f9221a8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:32.926346 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a84de-e19e-406e-b188-1a2f9221a8b9 None admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:32.926797 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a84de-e19e-406e-b188-1a2f9221a8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:32.927329 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7a84de-e19e-406e-b188-1a2f9221a8b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:32.928241 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7a84de-e19e-406e-b188-1a2f9221a8b9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:32.930042 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7a84de-e19e-406e-b188-1a2f9221a8b9 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:32.946835 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 21/38] 10.222.0.50 () {60 vars in 1131 bytes} [Fri May 27 03:08:32 2022] POST /identity/v3/roles => generated 225 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:08:32.954029 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a84de-e19e-406e-b188-1a2f9221a8b9 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:33.050648 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 18/39] 10.222.0.50 () {60 vars in 1141 bytes} [Fri May 27 03:08:32 2022] POST /identity/v3/projects => generated 301 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:08:33.051618 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ffc74-851a-4a0f-a76f-9f644d13bc69 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:33.096588 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7a84de-e19e-406e-b188-1a2f9221a8b9 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:33.100515 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:33.152355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cfafb2-92fe-4cae-9713-4e3f83b04302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:33.153215 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfafb2-92fe-4cae-9713-4e3f83b04302 None admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:33.153515 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfafb2-92fe-4cae-9713-4e3f83b04302 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:33.153798 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfafb2-92fe-4cae-9713-4e3f83b04302 None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:33.154566 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cfafb2-92fe-4cae-9713-4e3f83b04302 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:33.156017 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cfafb2-92fe-4cae-9713-4e3f83b04302 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:33.176358 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 22/40] 10.222.0.50 () {60 vars in 1131 bytes} [Fri May 27 03:08:32 2022] POST /identity/v3/roles => generated 223 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:08:33.178308 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ecf3c0-594d-4693-82dd-cea43ac9970d None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:33.179705 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ecf3c0-594d-4693-82dd-cea43ac9970d None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:33.180234 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ecf3c0-594d-4693-82dd-cea43ac9970d None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:33.182880 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 23/41] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:08:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:08:33.188699 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d276849-b512-4b2b-a416-807453a93cf9 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:33.189388 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d276849-b512-4b2b-a416-807453a93cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:33.189883 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d276849-b512-4b2b-a416-807453a93cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:33.215441 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-4d276849-b512-4b2b-a416-807453a93cf9 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:08:33.217950 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-4d276849-b512-4b2b-a416-807453a93cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:08:33.218120 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ffc74-851a-4a0f-a76f-9f644d13bc69 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:33.241072 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:33.264650 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ba65d-9db5-4e42-80a1-4402e36efaa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:33.265730 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ba65d-9db5-4e42-80a1-4402e36efaa7 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:33.266126 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ba65d-9db5-4e42-80a1-4402e36efaa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:33.266508 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ba65d-9db5-4e42-80a1-4402e36efaa7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:33.271541 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9ba65d-9db5-4e42-80a1-4402e36efaa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:33.273715 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9ba65d-9db5-4e42-80a1-4402e36efaa7 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:33.280205 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 19/42] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:08:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:33.283956 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ba65d-9db5-4e42-80a1-4402e36efaa7 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:33.338534 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 24/43] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:08:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:08:33.362368 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d276849-b512-4b2b-a416-807453a93cf9 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:33.419679 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:33.425750 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ba65d-9db5-4e42-80a1-4402e36efaa7 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:33.467119 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bddeec-4385-4e14-84d5-32dbd1d0e1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:33.468097 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bddeec-4385-4e14-84d5-32dbd1d0e1a0 None admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:33.468417 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bddeec-4385-4e14-84d5-32dbd1d0e1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:33.468661 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bddeec-4385-4e14-84d5-32dbd1d0e1a0 None admin] PATH_INFO: `/v3/domains/default/users/4a96d7acf93e46c7ac5700560db36dbe/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:33.470822 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bddeec-4385-4e14-84d5-32dbd1d0e1a0 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=4a96d7acf93e46c7ac5700560db36dbe, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:33.472302 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bddeec-4385-4e14-84d5-32dbd1d0e1a0 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:33.481546 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d276849-b512-4b2b-a416-807453a93cf9 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:33.488655 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 20/44] 10.222.0.50 () {58 vars in 1361 bytes} [Fri May 27 03:08:33 2022] PUT /identity/v3/domains/default/users/4a96d7acf93e46c7ac5700560db36dbe/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:08:33.520856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ba046-d18f-48a6-bad5-44305c4ee7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:33.521711 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ba046-d18f-48a6-bad5-44305c4ee7ba None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:33.521991 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ba046-d18f-48a6-bad5-44305c4ee7ba None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:33.522264 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ba046-d18f-48a6-bad5-44305c4ee7ba None admin] PATH_INFO: `/v3/domains/default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:33.523870 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ba046-d18f-48a6-bad5-44305c4ee7ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:33.525431 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ba046-d18f-48a6-bad5-44305c4ee7ba None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:33.527124 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 25/45] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:08:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:08:33.533231 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bddeec-4385-4e14-84d5-32dbd1d0e1a0 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:33.644757 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bddeec-4385-4e14-84d5-32dbd1d0e1a0 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:33.683682 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ba065e-5b86-4f98-a832-a82cb630d710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:33.684392 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba065e-5b86-4f98-a832-a82cb630d710 None admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:33.684742 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba065e-5b86-4f98-a832-a82cb630d710 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:33.685224 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba065e-5b86-4f98-a832-a82cb630d710 None admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:33.686039 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ba065e-5b86-4f98-a832-a82cb630d710 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:33.687918 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ba065e-5b86-4f98-a832-a82cb630d710 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:33.719159 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 21/46] 10.222.0.50 () {60 vars in 1141 bytes} [Fri May 27 03:08:33 2022] POST /identity/v3/projects => generated 312 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:08:35.666515 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e9a41-572f-47c7-a44e-2908d44ebb87 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:35.667026 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e9a41-572f-47c7-a44e-2908d44ebb87 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:35.667026 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e9a41-572f-47c7-a44e-2908d44ebb87 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:35.668325 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 26/47] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:08:35.674160 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d57b49-253a-45af-8a5d-c9801fe8cc5b None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:35.674557 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d57b49-253a-45af-8a5d-c9801fe8cc5b None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:35.674713 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d57b49-253a-45af-8a5d-c9801fe8cc5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:35.692981 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-45d57b49-253a-45af-8a5d-c9801fe8cc5b None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:08:35.694533 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-45d57b49-253a-45af-8a5d-c9801fe8cc5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:08:35.697695 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:35.772328 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 22/48] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:08:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:08:35.795710 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160e9a41-572f-47c7-a44e-2908d44ebb87 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:35.812417 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:35.904666 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160e9a41-572f-47c7-a44e-2908d44ebb87 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:35.940378 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205ad9c-ce17-42c5-8725-4cec562b7363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:35.941276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205ad9c-ce17-42c5-8725-4cec562b7363 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:35.941555 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205ad9c-ce17-42c5-8725-4cec562b7363 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:35.941831 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b205ad9c-ce17-42c5-8725-4cec562b7363 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:35.942847 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8bb8e2-69a9-4f1c-b173-3cc49549b898 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:35.943143 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8bb8e2-69a9-4f1c-b173-3cc49549b898 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:35.943427 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8bb8e2-69a9-4f1c-b173-3cc49549b898 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:35.944934 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 23/49] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:08:35.950235 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de35aa-a248-41c7-9b52-a1e26c5d02e1 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:35.950565 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de35aa-a248-41c7-9b52-a1e26c5d02e1 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:35.950857 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de35aa-a248-41c7-9b52-a1e26c5d02e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:35.953502 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b205ad9c-ce17-42c5-8725-4cec562b7363 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:08:35.963538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-b205ad9c-ce17-42c5-8725-4cec562b7363 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:08:35.963538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:08:35.963538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:08:35.963538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:08:35.963538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:08:35.963538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:08:35.963538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:08:35.963538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:08:35.963538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:08:35.963538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:08:35.963538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:08:35.963538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:08:35.963538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:08:35.963538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:08:35.963538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:08:35.963538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:08:35.964390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:08:35.964390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:08:35.964390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:08:35.964390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:08:35.964390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:08:35.964390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:08:35.964390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:08:35.964390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:08:35.964390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:08:35.964390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:08:35.964390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:08:35.964390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:08:35.964390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:08:35.964390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:08:35.964390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:08:35.964390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:08:35.965300 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:08:35.965300 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:08:35.965300 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:08:35.965300 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:08:35.965300 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:08:35.965300 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:08:35.965300 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:08:35.965300 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:08:35.965300 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:08:35.965300 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:08:35.965300 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:08:35.965300 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:08:35.965300 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 27/50] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 03:08:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:08:35.968626 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-80de35aa-a248-41c7-9b52-a1e26c5d02e1 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:08:35.970154 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-80de35aa-a248-41c7-9b52-a1e26c5d02e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:08:35.971077 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205ad9c-ce17-42c5-8725-4cec562b7363 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:36.037239 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 24/51] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:08:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:08:36.045094 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:36.060689 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80de35aa-a248-41c7-9b52-a1e26c5d02e1 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:36.076117 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b205ad9c-ce17-42c5-8725-4cec562b7363 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:36.110180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6972639e-f90f-4754-bf16-ef0f24f4a6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:36.110855 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972639e-f90f-4754-bf16-ef0f24f4a6c7 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:36.111141 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972639e-f90f-4754-bf16-ef0f24f4a6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:36.111420 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972639e-f90f-4754-bf16-ef0f24f4a6c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:36.112183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6972639e-f90f-4754-bf16-ef0f24f4a6c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:08:36.112494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6972639e-f90f-4754-bf16-ef0f24f4a6c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:36.113911 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6972639e-f90f-4754-bf16-ef0f24f4a6c7 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:36.126027 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 28/52] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:08:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:36.131759 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6972639e-f90f-4754-bf16-ef0f24f4a6c7 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:36.133913 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:36.164967 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80de35aa-a248-41c7-9b52-a1e26c5d02e1 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:36.199598 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e31a7c5-79fc-4443-9208-f48ef1e41bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:36.200313 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31a7c5-79fc-4443-9208-f48ef1e41bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:36.200593 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31a7c5-79fc-4443-9208-f48ef1e41bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:36.200862 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31a7c5-79fc-4443-9208-f48ef1e41bc5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:36.202097 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e31a7c5-79fc-4443-9208-f48ef1e41bc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:36.203557 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e31a7c5-79fc-4443-9208-f48ef1e41bc5 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:36.205411 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 25/53] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:08:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:08:36.211201 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e31a7c5-79fc-4443-9208-f48ef1e41bc5 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:36.232426 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6972639e-f90f-4754-bf16-ef0f24f4a6c7 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:36.265850 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069214bd-0033-40ab-84b2-b2e41a0271d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:36.266748 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069214bd-0033-40ab-84b2-b2e41a0271d3 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:36.267029 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069214bd-0033-40ab-84b2-b2e41a0271d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:36.267302 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069214bd-0033-40ab-84b2-b2e41a0271d3 None admin] PATH_INFO: `/v3/users/4a96d7acf93e46c7ac5700560db36dbe` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:36.270552 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069214bd-0033-40ab-84b2-b2e41a0271d3 None admin] RBAC: Authorizing `identity:get_user(user_id=4a96d7acf93e46c7ac5700560db36dbe)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:36.270552 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069214bd-0033-40ab-84b2-b2e41a0271d3 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:36.271724 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 29/54] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:08:36 2022] GET /identity/v3/users/4a96d7acf93e46c7ac5700560db36dbe => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:08:36.277104 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069214bd-0033-40ab-84b2-b2e41a0271d3 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:36.279840 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:36.314760 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e31a7c5-79fc-4443-9208-f48ef1e41bc5 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:36.349034 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa3c6a-8d9c-4972-bcab-619fe941bc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:36.349701 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa3c6a-8d9c-4972-bcab-619fe941bc3f None admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:36.349983 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa3c6a-8d9c-4972-bcab-619fe941bc3f None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:36.350258 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa3c6a-8d9c-4972-bcab-619fe941bc3f None admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:36.351121 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aa3c6a-8d9c-4972-bcab-619fe941bc3f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:36.352645 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aa3c6a-8d9c-4972-bcab-619fe941bc3f None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:36.360623 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:36.379110 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 26/55] 10.222.0.50 () {60 vars in 1140 bytes} [Fri May 27 03:08:36 2022] POST /identity/v3/projects => generated 298 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:08:36.382281 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069214bd-0033-40ab-84b2-b2e41a0271d3 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:36.421577 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004dae6e-e575-438f-87ee-922115e5f102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:36.422403 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004dae6e-e575-438f-87ee-922115e5f102 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:36.422680 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004dae6e-e575-438f-87ee-922115e5f102 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:36.422935 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004dae6e-e575-438f-87ee-922115e5f102 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:36.424226 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004dae6e-e575-438f-87ee-922115e5f102 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:36.425755 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004dae6e-e575-438f-87ee-922115e5f102 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:36.427412 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 30/56] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:08:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:36.434123 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa3c6a-8d9c-4972-bcab-619fe941bc3f None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:36.545043 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa3c6a-8d9c-4972-bcab-619fe941bc3f None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:36.580779 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b4e4d9-8de5-44a4-874e-1557bda5588c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:36.581526 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b4e4d9-8de5-44a4-874e-1557bda5588c None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:36.581816 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b4e4d9-8de5-44a4-874e-1557bda5588c None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:36.582090 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b4e4d9-8de5-44a4-874e-1557bda5588c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:36.582938 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b4e4d9-8de5-44a4-874e-1557bda5588c None admin] RBAC: Adding query filter params (role.id=b4c524dd7bb44e94bd3b42d86e1cdf42, scope.domain.id=default, user.id=4a96d7acf93e46c7ac5700560db36dbe) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:08:36.583244 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b4e4d9-8de5-44a4-874e-1557bda5588c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:36.584718 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b4e4d9-8de5-44a4-874e-1557bda5588c None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:36.591528 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 27/57] 10.222.0.50 () {56 vars in 1322 bytes} [Fri May 27 03:08:36 2022] GET /identity/v3/role_assignments?user.id=4a96d7acf93e46c7ac5700560db36dbe&scope.domain.id=default&role.id=b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 530 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:38.576500 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eea031-f60d-4c57-9df2-6468ae4cbec3 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:38.577350 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eea031-f60d-4c57-9df2-6468ae4cbec3 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:38.577350 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85eea031-f60d-4c57-9df2-6468ae4cbec3 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:38.579498 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 31/58] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:08:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:08:38.585780 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd60e4-4b55-4274-a785-699dbd852f1b None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:38.586223 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd60e4-4b55-4274-a785-699dbd852f1b None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:38.586392 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dd60e4-4b55-4274-a785-699dbd852f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:38.609859 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-d4dd60e4-4b55-4274-a785-699dbd852f1b None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:08:38.611768 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-d4dd60e4-4b55-4274-a785-699dbd852f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:08:38.669551 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:38.690824 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 28/59] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:08:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:08:38.715250 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eea031-f60d-4c57-9df2-6468ae4cbec3 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:38.792239 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:38.826771 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85eea031-f60d-4c57-9df2-6468ae4cbec3 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:38.862187 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a65b7d6-f4f2-430e-8a82-da51daeb9490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:38.862988 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65b7d6-f4f2-430e-8a82-da51daeb9490 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:38.863269 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65b7d6-f4f2-430e-8a82-da51daeb9490 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:38.863543 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65b7d6-f4f2-430e-8a82-da51daeb9490 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:38.864853 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a65b7d6-f4f2-430e-8a82-da51daeb9490 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:38.866499 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a65b7d6-f4f2-430e-8a82-da51daeb9490 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:38.868111 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 32/60] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:08:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:38.876821 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd60e4-4b55-4274-a785-699dbd852f1b None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:38.989541 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dd60e4-4b55-4274-a785-699dbd852f1b None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:39.030735 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcef8280-b376-43ff-9eb1-f1730eb84fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:39.031402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef8280-b376-43ff-9eb1-f1730eb84fb8 None admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:39.031683 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef8280-b376-43ff-9eb1-f1730eb84fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:39.031959 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef8280-b376-43ff-9eb1-f1730eb84fb8 None admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:39.032662 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcef8280-b376-43ff-9eb1-f1730eb84fb8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:39.034150 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcef8280-b376-43ff-9eb1-f1730eb84fb8 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:39.063248 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 29/61] 10.222.0.50 () {60 vars in 1132 bytes} [Fri May 27 03:08:38 2022] POST /identity/v3/users => generated 275 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:08:42.470880 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d759789-808f-4523-8a79-e1e591a03cec None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:42.471967 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d759789-808f-4523-8a79-e1e591a03cec None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:42.472449 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d759789-808f-4523-8a79-e1e591a03cec None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:42.474099 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 33/62] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:08:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:08:42.479795 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766a4c4-49e3-413b-a69d-d8532f46ecd1 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:42.480219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766a4c4-49e3-413b-a69d-d8532f46ecd1 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:42.480385 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766a4c4-49e3-413b-a69d-d8532f46ecd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:42.505972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-6766a4c4-49e3-413b-a69d-d8532f46ecd1 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:08:42.507217 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-6766a4c4-49e3-413b-a69d-d8532f46ecd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:08:42.530526 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb82e61-d67d-47ce-8503-449dd7f3bace None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:42.531138 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb82e61-d67d-47ce-8503-449dd7f3bace None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:42.531603 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb82e61-d67d-47ce-8503-449dd7f3bace None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:42.533187 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 34/63] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:08:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:08:42.538569 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63515a4-0509-463a-b6b1-f94f829cb6b4 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:42.539155 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63515a4-0509-463a-b6b1-f94f829cb6b4 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:42.539628 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63515a4-0509-463a-b6b1-f94f829cb6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:42.550628 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:42.572815 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-e63515a4-0509-463a-b6b1-f94f829cb6b4 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:08:42.574404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-e63515a4-0509-463a-b6b1-f94f829cb6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:08:42.599261 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 30/64] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:08:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:08:42.614342 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b71356-bf58-4b8d-9d4d-a4c384918a73 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:42.614624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b71356-bf58-4b8d-9d4d-a4c384918a73 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:42.614902 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b71356-bf58-4b8d-9d4d-a4c384918a73 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:42.616275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 31/65] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:08:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:08:42.625959 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ee2a3-c855-4233-8429-ad953c3bb0a5 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:42.626246 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ee2a3-c855-4233-8429-ad953c3bb0a5 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:42.626524 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ee2a3-c855-4233-8429-ad953c3bb0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:42.648042 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-fd5ee2a3-c855-4233-8429-ad953c3bb0a5 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:08:42.650033 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-fd5ee2a3-c855-4233-8429-ad953c3bb0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:08:42.669114 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 35/66] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:08:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:08:42.670059 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63515a4-0509-463a-b6b1-f94f829cb6b4 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:42.745997 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 32/67] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:08:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:08:42.746908 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ee2a3-c855-4233-8429-ad953c3bb0a5 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:42.753486 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:42.792057 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63515a4-0509-463a-b6b1-f94f829cb6b4 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:42.836253 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd37227f-0519-47bb-8caa-1dcd2d435f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:42.837567 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd37227f-0519-47bb-8caa-1dcd2d435f93 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:42.838252 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd37227f-0519-47bb-8caa-1dcd2d435f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:42.839059 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd37227f-0519-47bb-8caa-1dcd2d435f93 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:42.853141 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd37227f-0519-47bb-8caa-1dcd2d435f93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:08:42.863451 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-fd37227f-0519-47bb-8caa-1dcd2d435f93 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:08:42.863451 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:08:42.863451 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:08:42.863451 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:08:42.863451 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:08:42.863451 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:08:42.863451 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:08:42.863451 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:08:42.863451 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:08:42.863451 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:08:42.863451 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:08:42.863451 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:08:42.863451 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:08:42.863451 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:08:42.863451 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:08:42.863451 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:08:42.864491 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:08:42.864491 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:08:42.864491 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:08:42.864491 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:08:42.864491 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:08:42.864491 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:08:42.864491 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:08:42.864491 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:08:42.864491 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:08:42.864491 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:08:42.864491 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:08:42.864491 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:08:42.864491 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:08:42.864491 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:08:42.864491 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:08:42.864491 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:08:42.865369 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:08:42.865369 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:08:42.865369 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:08:42.865369 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:08:42.865369 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:08:42.865369 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:08:42.865369 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:08:42.865369 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:08:42.865369 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:08:42.865369 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:08:42.865369 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:08:42.865369 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:08:42.866313 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 36/68] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 03:08:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:08:42.868020 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd37227f-0519-47bb-8caa-1dcd2d435f93 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:42.885878 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:42.890731 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ee2a3-c855-4233-8429-ad953c3bb0a5 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:42.934283 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc8684-ce11-481a-81c5-0f3819df5f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:42.935391 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc8684-ce11-481a-81c5-0f3819df5f3f None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:42.935938 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc8684-ce11-481a-81c5-0f3819df5f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:42.936364 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc8684-ce11-481a-81c5-0f3819df5f3f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:42.950414 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfc8684-ce11-481a-81c5-0f3819df5f3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:08:42.961653 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-6dfc8684-ce11-481a-81c5-0f3819df5f3f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:08:42.961653 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:08:42.961653 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:08:42.961653 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:08:42.961653 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:08:42.961653 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:08:42.961653 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:08:42.961653 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:08:42.961653 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:08:42.961653 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:08:42.961653 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:08:42.961653 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:08:42.961653 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:08:42.961653 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:08:42.961653 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:08:42.961653 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:08:42.962599 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:08:42.962599 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:08:42.962599 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:08:42.962599 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:08:42.962599 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:08:42.962599 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:08:42.962599 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:08:42.962599 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:08:42.962599 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:08:42.962599 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:08:42.962599 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:08:42.962599 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:08:42.962599 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:08:42.962599 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:08:42.962599 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:08:42.962599 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:08:42.963359 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:08:42.963359 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:08:42.963359 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:08:42.963359 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:08:42.963359 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:08:42.963359 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:08:42.963359 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:08:42.963359 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:08:42.963359 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:08:42.963359 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:08:42.963359 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:08:42.963359 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:08:42.963359 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 33/69] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 03:08:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:08:42.963982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc8684-ce11-481a-81c5-0f3819df5f3f None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:42.993753 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd37227f-0519-47bb-8caa-1dcd2d435f93 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:43.023932 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:43.033459 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc7d89-9475-43cb-ae49-0dc8d6953dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:43.034067 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc7d89-9475-43cb-ae49-0dc8d6953dbf None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:43.034350 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc7d89-9475-43cb-ae49-0dc8d6953dbf None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:43.034636 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc7d89-9475-43cb-ae49-0dc8d6953dbf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:43.046449 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc7d89-9475-43cb-ae49-0dc8d6953dbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:08:43.054190 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-4adc7d89-9475-43cb-ae49-0dc8d6953dbf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:08:43.054190 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:08:43.054190 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:08:43.054190 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:08:43.054190 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:08:43.054190 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:08:43.054190 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:08:43.054190 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:08:43.054190 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:08:43.054190 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:08:43.054190 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:08:43.054190 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:08:43.054190 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:08:43.054190 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:08:43.054190 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:08:43.054190 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:08:43.055060 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:08:43.055060 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:08:43.055060 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:08:43.055060 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:08:43.055060 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:08:43.055060 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:08:43.055060 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:08:43.055060 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:08:43.055060 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:08:43.055060 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:08:43.055060 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:08:43.055060 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:08:43.055060 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:08:43.055060 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:08:43.055060 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:08:43.055060 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:08:43.055802 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:08:43.055802 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:08:43.055802 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:08:43.055802 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:08:43.055802 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:08:43.055802 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:08:43.055802 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:08:43.055802 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:08:43.055802 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:08:43.055802 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:08:43.055802 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:08:43.055802 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:08:43.055802 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 37/70] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:08:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:08:43.056912 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb855e7d-3cb5-4535-af0c-b183bd8eaf6e None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:43.057141 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb855e7d-3cb5-4535-af0c-b183bd8eaf6e None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:43.057426 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb855e7d-3cb5-4535-af0c-b183bd8eaf6e None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:43.058693 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 38/71] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:08:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:08:43.063024 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb855e7d-3cb5-4535-af0c-b183bd8eaf6e None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:43.087854 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc8684-ce11-481a-81c5-0f3819df5f3f None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:43.121306 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843978d3-a90f-4de0-a0ab-a112463f40a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:43.121986 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843978d3-a90f-4de0-a0ab-a112463f40a5 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:43.122280 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843978d3-a90f-4de0-a0ab-a112463f40a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:43.122562 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843978d3-a90f-4de0-a0ab-a112463f40a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:43.123318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843978d3-a90f-4de0-a0ab-a112463f40a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:08:43.123630 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843978d3-a90f-4de0-a0ab-a112463f40a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:43.125126 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843978d3-a90f-4de0-a0ab-a112463f40a5 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:43.134091 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 34/72] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:08:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:43.134961 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843978d3-a90f-4de0-a0ab-a112463f40a5 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:43.159749 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:43.175324 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb855e7d-3cb5-4535-af0c-b183bd8eaf6e None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:43.209307 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb9ea4e-05b0-4be6-9714-9f428cc2bdd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:43.209980 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb9ea4e-05b0-4be6-9714-9f428cc2bdd5 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:43.210263 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb9ea4e-05b0-4be6-9714-9f428cc2bdd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:43.210616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb9ea4e-05b0-4be6-9714-9f428cc2bdd5 None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:43.211473 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb9ea4e-05b0-4be6-9714-9f428cc2bdd5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:08:43.211779 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb9ea4e-05b0-4be6-9714-9f428cc2bdd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:43.213218 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb9ea4e-05b0-4be6-9714-9f428cc2bdd5 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:43.221350 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 39/73] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:08:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:43.222628 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2206d6a8-fd19-4fc7-9912-71d08c84835c None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:43.222922 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2206d6a8-fd19-4fc7-9912-71d08c84835c None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:43.223195 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2206d6a8-fd19-4fc7-9912-71d08c84835c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:43.240080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-2206d6a8-fd19-4fc7-9912-71d08c84835c None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:08:43.241315 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-2206d6a8-fd19-4fc7-9912-71d08c84835c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:08:43.246948 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843978d3-a90f-4de0-a0ab-a112463f40a5 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:43.281314 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa0f1e1-e5c6-4406-b774-83fa5d77ab77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:43.281993 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0f1e1-e5c6-4406-b774-83fa5d77ab77 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:43.282278 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0f1e1-e5c6-4406-b774-83fa5d77ab77 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:43.282557 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0f1e1-e5c6-4406-b774-83fa5d77ab77 None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:43.283321 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa0f1e1-e5c6-4406-b774-83fa5d77ab77 None admin] RBAC: Adding query filter params (name=member) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:08:43.283632 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa0f1e1-e5c6-4406-b774-83fa5d77ab77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:43.285062 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa0f1e1-e5c6-4406-b774-83fa5d77ab77 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:43.289338 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:43.299302 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 35/74] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 03:08:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:43.299302 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa0f1e1-e5c6-4406-b774-83fa5d77ab77 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:43.312989 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 40/75] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:08:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:08:43.314458 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13592ca6-d739-4eb7-af66-2361e80e7848 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:43.314752 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13592ca6-d739-4eb7-af66-2361e80e7848 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:43.315029 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13592ca6-d739-4eb7-af66-2361e80e7848 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:43.316292 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 41/76] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:08:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:08:43.317093 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13592ca6-d739-4eb7-af66-2361e80e7848 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:43.390166 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:43.406199 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa0f1e1-e5c6-4406-b774-83fa5d77ab77 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:43.419505 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13592ca6-d739-4eb7-af66-2361e80e7848 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:43.440456 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d57a9-7533-4101-a7c9-bab3fa31ae2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:43.441373 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d57a9-7533-4101-a7c9-bab3fa31ae2d None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:43.441649 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d57a9-7533-4101-a7c9-bab3fa31ae2d None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:43.441929 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d57a9-7533-4101-a7c9-bab3fa31ae2d None admin] PATH_INFO: `/v3/users/4a96d7acf93e46c7ac5700560db36dbe` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:43.443117 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d57a9-7533-4101-a7c9-bab3fa31ae2d None admin] RBAC: Authorizing `identity:get_user(user_id=4a96d7acf93e46c7ac5700560db36dbe)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:43.444605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d57a9-7533-4101-a7c9-bab3fa31ae2d None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:43.446548 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 36/77] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:08:43 2022] GET /identity/v3/users/4a96d7acf93e46c7ac5700560db36dbe => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:43.447314 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d57a9-7533-4101-a7c9-bab3fa31ae2d None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:43.455948 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d240e734-6cf2-43ef-af20-81f5c13e4e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:43.456825 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240e734-6cf2-43ef-af20-81f5c13e4e41 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:43.457146 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240e734-6cf2-43ef-af20-81f5c13e4e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:43.457418 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d240e734-6cf2-43ef-af20-81f5c13e4e41 None admin] PATH_INFO: `/v3/users/ba8ddcedde1a4bee9ba699929814f673` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:43.472477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d240e734-6cf2-43ef-af20-81f5c13e4e41 None admin] RBAC: Authorizing `identity:get_user(user_id=ba8ddcedde1a4bee9ba699929814f673)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:43.473982 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d240e734-6cf2-43ef-af20-81f5c13e4e41 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:43.475682 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 42/78] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:08:43 2022] GET /identity/v3/users/ba8ddcedde1a4bee9ba699929814f673 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:43.476970 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004da719-c342-4965-8cb4-b5318d225cbf None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:43.477258 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004da719-c342-4965-8cb4-b5318d225cbf None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:43.477536 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004da719-c342-4965-8cb4-b5318d225cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:43.493550 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-004da719-c342-4965-8cb4-b5318d225cbf None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:08:43.494780 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-004da719-c342-4965-8cb4-b5318d225cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:08:43.526705 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:43.548688 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d57a9-7533-4101-a7c9-bab3fa31ae2d None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:43.565630 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 43/79] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:08:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:08:43.566473 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004da719-c342-4965-8cb4-b5318d225cbf None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:43.588175 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6fd97-6d5d-4aa0-bcab-3b5c73616fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:43.589085 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6fd97-6d5d-4aa0-bcab-3b5c73616fd2 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:43.589370 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6fd97-6d5d-4aa0-bcab-3b5c73616fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:43.589649 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6fd97-6d5d-4aa0-bcab-3b5c73616fd2 None admin] PATH_INFO: `/v3/users/ba8ddcedde1a4bee9ba699929814f673` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:43.590847 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac6fd97-6d5d-4aa0-bcab-3b5c73616fd2 None admin] RBAC: Authorizing `identity:get_user(user_id=ba8ddcedde1a4bee9ba699929814f673)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:43.592340 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac6fd97-6d5d-4aa0-bcab-3b5c73616fd2 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:43.594023 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 37/80] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:08:43 2022] GET /identity/v3/users/ba8ddcedde1a4bee9ba699929814f673 => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:43.594797 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6fd97-6d5d-4aa0-bcab-3b5c73616fd2 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:43.618287 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:43.670994 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004da719-c342-4965-8cb4-b5318d225cbf None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:43.702983 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6fd97-6d5d-4aa0-bcab-3b5c73616fd2 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:43.707433 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc82eea-81df-40f7-ace5-b90035a0c61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:43.708245 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc82eea-81df-40f7-ace5-b90035a0c61d None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:43.708526 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc82eea-81df-40f7-ace5-b90035a0c61d None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:43.708807 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc82eea-81df-40f7-ace5-b90035a0c61d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:43.719446 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc82eea-81df-40f7-ace5-b90035a0c61d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:08:43.727059 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-bcc82eea-81df-40f7-ace5-b90035a0c61d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:08:43.727059 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:08:43.727059 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:08:43.727059 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:08:43.727059 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:08:43.727059 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:08:43.727059 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:08:43.727059 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:08:43.727059 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:08:43.727059 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:08:43.727059 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:08:43.727059 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:08:43.727059 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:08:43.727059 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:08:43.727059 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:08:43.727059 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:08:43.728078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:08:43.728078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:08:43.728078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:08:43.728078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:08:43.728078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:08:43.728078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:08:43.728078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:08:43.728078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:08:43.728078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:08:43.728078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:08:43.728078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:08:43.728078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:08:43.728078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:08:43.728078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:08:43.728078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:08:43.728078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:08:43.729179 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:08:43.729179 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:08:43.729179 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:08:43.729179 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:08:43.729179 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:08:43.729179 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:08:43.729179 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:08:43.729179 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:08:43.729179 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:08:43.729179 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:08:43.729179 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:08:43.729179 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:08:43.729179 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 44/81] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:08:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:08:43.729997 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc82eea-81df-40f7-ace5-b90035a0c61d None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:43.748831 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a3758-c878-4194-9b97-5c05c37b8a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:43.749567 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a3758-c878-4194-9b97-5c05c37b8a1a None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:43.749884 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a3758-c878-4194-9b97-5c05c37b8a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:43.750192 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a3758-c878-4194-9b97-5c05c37b8a1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:43.758298 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:43.820706 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a3758-c878-4194-9b97-5c05c37b8a1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:43.822487 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a3758-c878-4194-9b97-5c05c37b8a1a None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:43.829221 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc82eea-81df-40f7-ace5-b90035a0c61d None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:43.842700 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:43.862638 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f85fa-de9f-4a2b-b927-1ea4d3dd8acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:43.863383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f85fa-de9f-4a2b-b927-1ea4d3dd8acc None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:43.863675 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f85fa-de9f-4a2b-b927-1ea4d3dd8acc None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:43.863946 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f85fa-de9f-4a2b-b927-1ea4d3dd8acc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:43.922039 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 38/82] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:08:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:08:43.922712 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a3758-c878-4194-9b97-5c05c37b8a1a None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:43.934096 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556f85fa-de9f-4a2b-b927-1ea4d3dd8acc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:43.935608 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556f85fa-de9f-4a2b-b927-1ea4d3dd8acc None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:43.963406 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:44.033112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a3758-c878-4194-9b97-5c05c37b8a1a None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:44.044365 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 45/83] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:08:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:08:44.045250 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f85fa-de9f-4a2b-b927-1ea4d3dd8acc None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:44.058520 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:44.073721 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5911b481-4891-46fc-a020-1defbd3f496c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:44.074588 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911b481-4891-46fc-a020-1defbd3f496c None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:44.074837 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911b481-4891-46fc-a020-1defbd3f496c None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:44.075112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911b481-4891-46fc-a020-1defbd3f496c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:44.076326 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5911b481-4891-46fc-a020-1defbd3f496c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:44.077871 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5911b481-4891-46fc-a020-1defbd3f496c None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:44.079553 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 39/84] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:08:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:44.080318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5911b481-4891-46fc-a020-1defbd3f496c None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:44.141658 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f85fa-de9f-4a2b-b927-1ea4d3dd8acc None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:44.168851 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:44.177188 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb4ae68-3fd0-46be-b4bb-b62737f79a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:44.177780 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb4ae68-3fd0-46be-b4bb-b62737f79a84 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:44.178066 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb4ae68-3fd0-46be-b4bb-b62737f79a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:44.178343 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb4ae68-3fd0-46be-b4bb-b62737f79a84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:44.179390 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5911b481-4891-46fc-a020-1defbd3f496c None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:44.212568 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f00a9-958e-48ef-b457-510f726e9229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:44.213339 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f00a9-958e-48ef-b457-510f726e9229 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:44.213624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f00a9-958e-48ef-b457-510f726e9229 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:44.213899 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f00a9-958e-48ef-b457-510f726e9229 None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:44.214643 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493f00a9-958e-48ef-b457-510f726e9229 None admin] RBAC: Adding query filter params (name=member) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:08:44.214949 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493f00a9-958e-48ef-b457-510f726e9229 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:44.216347 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493f00a9-958e-48ef-b457-510f726e9229 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:44.224525 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 40/85] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 03:08:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:44.225402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f00a9-958e-48ef-b457-510f726e9229 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:44.244779 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb4ae68-3fd0-46be-b4bb-b62737f79a84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:44.246309 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb4ae68-3fd0-46be-b4bb-b62737f79a84 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:44.275999 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:44.328562 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f00a9-958e-48ef-b457-510f726e9229 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:44.344630 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 46/86] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:08:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:08:44.345532 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb4ae68-3fd0-46be-b4bb-b62737f79a84 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:44.362088 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c9ab4a-76ab-4413-923a-fefce36a2d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:44.362985 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9ab4a-76ab-4413-923a-fefce36a2d34 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:44.363270 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9ab4a-76ab-4413-923a-fefce36a2d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:44.363541 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9ab4a-76ab-4413-923a-fefce36a2d34 None admin] PATH_INFO: `/v3/projects/4edef8e255584377bf94a9ac9712b4b7` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:44.364756 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c9ab4a-76ab-4413-923a-fefce36a2d34 None admin] RBAC: Authorizing `identity:get_project(project_id=4edef8e255584377bf94a9ac9712b4b7)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:44.366220 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c9ab4a-76ab-4413-923a-fefce36a2d34 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:44.368112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 41/87] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:08:44 2022] GET /identity/v3/projects/4edef8e255584377bf94a9ac9712b4b7 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:44.368858 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c9ab4a-76ab-4413-923a-fefce36a2d34 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:44.377592 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:44.460599 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb4ae68-3fd0-46be-b4bb-b62737f79a84 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:44.470064 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c9ab4a-76ab-4413-923a-fefce36a2d34 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:44.486679 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:44.495946 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d05d03a-2088-4c4c-9c3e-85136da1912b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:44.496826 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d05d03a-2088-4c4c-9c3e-85136da1912b None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:44.497138 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d05d03a-2088-4c4c-9c3e-85136da1912b None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:44.497418 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d05d03a-2088-4c4c-9c3e-85136da1912b None admin] PATH_INFO: `/v3/projects/4edef8e255584377bf94a9ac9712b4b7` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:44.498605 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d05d03a-2088-4c4c-9c3e-85136da1912b None admin] RBAC: Authorizing `identity:get_project(project_id=4edef8e255584377bf94a9ac9712b4b7)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:44.500086 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d05d03a-2088-4c4c-9c3e-85136da1912b None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:44.501815 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 47/88] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:08:44 2022] GET /identity/v3/projects/4edef8e255584377bf94a9ac9712b4b7 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:44.502574 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d05d03a-2088-4c4c-9c3e-85136da1912b None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:44.507431 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28b4e7-85bc-49a7-8b18-8a1e4d0a85a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:44.508129 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28b4e7-85bc-49a7-8b18-8a1e4d0a85a2 None admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:44.508446 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28b4e7-85bc-49a7-8b18-8a1e4d0a85a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:44.508726 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28b4e7-85bc-49a7-8b18-8a1e4d0a85a2 None admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:44.509572 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf28b4e7-85bc-49a7-8b18-8a1e4d0a85a2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:44.511038 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf28b4e7-85bc-49a7-8b18-8a1e4d0a85a2 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:44.532581 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 42/89] 10.222.0.50 () {60 vars in 1132 bytes} [Fri May 27 03:08:44 2022] POST /identity/v3/users => generated 289 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:08:44.533468 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28b4e7-85bc-49a7-8b18-8a1e4d0a85a2 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:44.635994 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:44.645190 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d05d03a-2088-4c4c-9c3e-85136da1912b None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:44.651583 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28b4e7-85bc-49a7-8b18-8a1e4d0a85a2 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:44.686916 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3583fdbe-1320-43ab-9da4-af58d8488d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:44.687829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583fdbe-1320-43ab-9da4-af58d8488d88 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:44.688188 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583fdbe-1320-43ab-9da4-af58d8488d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:44.688388 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583fdbe-1320-43ab-9da4-af58d8488d88 None admin] PATH_INFO: `/v3/users/ba8ddcedde1a4bee9ba699929814f673` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:44.689257 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58336a5a-e3dd-45d2-bbc6-5d0f2e8ed537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:44.689595 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3583fdbe-1320-43ab-9da4-af58d8488d88 None admin] RBAC: Authorizing `identity:get_user(user_id=ba8ddcedde1a4bee9ba699929814f673)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:44.690147 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58336a5a-e3dd-45d2-bbc6-5d0f2e8ed537 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:44.690431 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58336a5a-e3dd-45d2-bbc6-5d0f2e8ed537 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:44.690709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58336a5a-e3dd-45d2-bbc6-5d0f2e8ed537 None admin] PATH_INFO: `/v3/projects/4edef8e255584377bf94a9ac9712b4b7` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:44.691097 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3583fdbe-1320-43ab-9da4-af58d8488d88 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:44.691984 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58336a5a-e3dd-45d2-bbc6-5d0f2e8ed537 None admin] RBAC: Authorizing `identity:get_project(project_id=4edef8e255584377bf94a9ac9712b4b7)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:44.697590 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 48/90] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:08:44 2022] GET /identity/v3/users/ba8ddcedde1a4bee9ba699929814f673 => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:44.697763 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58336a5a-e3dd-45d2-bbc6-5d0f2e8ed537 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:44.697838 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3583fdbe-1320-43ab-9da4-af58d8488d88 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:44.697903 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 43/91] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:08:44 2022] GET /identity/v3/projects/4edef8e255584377bf94a9ac9712b4b7 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:44.697903 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58336a5a-e3dd-45d2-bbc6-5d0f2e8ed537 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:44.749464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:44.819279 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3583fdbe-1320-43ab-9da4-af58d8488d88 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:44.822973 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58336a5a-e3dd-45d2-bbc6-5d0f2e8ed537 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:44.863961 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1710b00e-1f5e-465d-bca6-b2218839670b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:44.864174 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710b00e-1f5e-465d-bca6-b2218839670b None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:44.864286 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710b00e-1f5e-465d-bca6-b2218839670b None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:44.864888 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710b00e-1f5e-465d-bca6-b2218839670b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:44.865433 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1710b00e-1f5e-465d-bca6-b2218839670b None admin] RBAC: Adding query filter params (role.id=b4c524dd7bb44e94bd3b42d86e1cdf42, scope.project.id=4edef8e255584377bf94a9ac9712b4b7, user.id=4a96d7acf93e46c7ac5700560db36dbe) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:08:44.866792 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ba557-59a3-4b52-aa30-5e390be4cf91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:44.867473 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ba557-59a3-4b52-aa30-5e390be4cf91 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:44.867757 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ba557-59a3-4b52-aa30-5e390be4cf91 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:44.868034 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ba557-59a3-4b52-aa30-5e390be4cf91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:44.868878 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712ba557-59a3-4b52-aa30-5e390be4cf91 None admin] RBAC: Adding query filter params (role.id=9e63b73b3ce24bb3ad214ecdb6132696, scope.project.id=4edef8e255584377bf94a9ac9712b4b7, user.id=ba8ddcedde1a4bee9ba699929814f673) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:08:44.869279 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712ba557-59a3-4b52-aa30-5e390be4cf91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:44.870779 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712ba557-59a3-4b52-aa30-5e390be4cf91 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:44.871174 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1710b00e-1f5e-465d-bca6-b2218839670b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:44.872859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1710b00e-1f5e-465d-bca6-b2218839670b None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:44.877652 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 44/92] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:08:44 2022] GET /identity/v3/role_assignments?user.id=ba8ddcedde1a4bee9ba699929814f673&scope.project.id=4edef8e255584377bf94a9ac9712b4b7&role.id=9e63b73b3ce24bb3ad214ecdb6132696 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:44.878501 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ba557-59a3-4b52-aa30-5e390be4cf91 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:44.882716 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 49/93] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:08:44 2022] GET /identity/v3/role_assignments?user.id=4a96d7acf93e46c7ac5700560db36dbe&scope.project.id=4edef8e255584377bf94a9ac9712b4b7&role.id=b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:44.884630 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1710b00e-1f5e-465d-bca6-b2218839670b None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:44.958307 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:45.014189 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ba557-59a3-4b52-aa30-5e390be4cf91 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:45.024204 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:45.061025 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d9116d-48b8-4573-88e9-ab015b9b9d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:45.061830 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d9116d-48b8-4573-88e9-ab015b9b9d65 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:45.062181 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d9116d-48b8-4573-88e9-ab015b9b9d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:45.062529 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d9116d-48b8-4573-88e9-ab015b9b9d65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:45.150679 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d9116d-48b8-4573-88e9-ab015b9b9d65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:45.152603 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d9116d-48b8-4573-88e9-ab015b9b9d65 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:45.163156 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1710b00e-1f5e-465d-bca6-b2218839670b None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:45.251832 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9bf95e-4c1b-496d-af5c-542556c93093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:45.252816 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9bf95e-4c1b-496d-af5c-542556c93093 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:45.253296 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9bf95e-4c1b-496d-af5c-542556c93093 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:45.253744 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9bf95e-4c1b-496d-af5c-542556c93093 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:45.254788 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9bf95e-4c1b-496d-af5c-542556c93093 None admin] RBAC: Adding query filter params (role.id=0129dd45ff5449f9957e8efdd01a55ed, scope.project.id=4edef8e255584377bf94a9ac9712b4b7, user.id=ba8ddcedde1a4bee9ba699929814f673) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:08:45.255276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9bf95e-4c1b-496d-af5c-542556c93093 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:45.256926 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9bf95e-4c1b-496d-af5c-542556c93093 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:45.263482 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 50/94] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:08:44 2022] GET /identity/v3/role_assignments?user.id=ba8ddcedde1a4bee9ba699929814f673&scope.project.id=4edef8e255584377bf94a9ac9712b4b7&role.id=0129dd45ff5449f9957e8efdd01a55ed => generated 259 bytes in 380 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:45.272037 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 45/95] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:08:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:08:45.282418 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9bf95e-4c1b-496d-af5c-542556c93093 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:45.415046 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:45.430450 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9bf95e-4c1b-496d-af5c-542556c93093 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:45.472567 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66419a74-b8de-47cd-a77b-730f60cc9b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:45.473491 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66419a74-b8de-47cd-a77b-730f60cc9b25 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:45.473771 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66419a74-b8de-47cd-a77b-730f60cc9b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:45.474040 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66419a74-b8de-47cd-a77b-730f60cc9b25 None admin] PATH_INFO: `/v3/projects/0f4c723f76b04fab9f9ac3617cf3fb8b` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:45.475292 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66419a74-b8de-47cd-a77b-730f60cc9b25 None admin] RBAC: Authorizing `identity:get_project(project_id=0f4c723f76b04fab9f9ac3617cf3fb8b)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:45.476787 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66419a74-b8de-47cd-a77b-730f60cc9b25 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:45.478837 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 51/96] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:08:45 2022] GET /identity/v3/projects/0f4c723f76b04fab9f9ac3617cf3fb8b => generated 312 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:45.484771 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d9116d-48b8-4573-88e9-ab015b9b9d65 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:45.493594 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:45.608090 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d9116d-48b8-4573-88e9-ab015b9b9d65 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:45.645535 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ffa18-88c3-4365-8b93-42e06d1c1f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:45.646202 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ffa18-88c3-4365-8b93-42e06d1c1f32 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:45.646488 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ffa18-88c3-4365-8b93-42e06d1c1f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:45.646757 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ffa18-88c3-4365-8b93-42e06d1c1f32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:45.648756 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97ffa18-88c3-4365-8b93-42e06d1c1f32 None admin] RBAC: Adding query filter params (role.id=0129dd45ff5449f9957e8efdd01a55ed, scope.project.id=0f4c723f76b04fab9f9ac3617cf3fb8b, user.id=ba8ddcedde1a4bee9ba699929814f673) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:08:45.648756 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97ffa18-88c3-4365-8b93-42e06d1c1f32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:45.649611 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97ffa18-88c3-4365-8b93-42e06d1c1f32 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:45.655733 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 46/97] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:08:45 2022] GET /identity/v3/role_assignments?user.id=ba8ddcedde1a4bee9ba699929814f673&scope.project.id=0f4c723f76b04fab9f9ac3617cf3fb8b&role.id=0129dd45ff5449f9957e8efdd01a55ed => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:47.697623 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923703b-480e-4a5a-8d5a-ecf37a28c278 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:47.698190 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923703b-480e-4a5a-8d5a-ecf37a28c278 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:47.698190 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3923703b-480e-4a5a-8d5a-ecf37a28c278 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:47.699664 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 52/98] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:08:47.701139 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec342a84-0824-4f12-875d-b9f148e9ac6a None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:47.701502 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec342a84-0824-4f12-875d-b9f148e9ac6a None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:47.701745 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec342a84-0824-4f12-875d-b9f148e9ac6a None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:47.703366 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 47/99] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:08:47.708678 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcf520-12dd-4070-b37d-5abcfbd3853e None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:47.709011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcf520-12dd-4070-b37d-5abcfbd3853e None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:47.709295 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcf520-12dd-4070-b37d-5abcfbd3853e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:47.724297 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57304df7-098e-42db-b6e9-53eed4316b01 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:47.724631 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57304df7-098e-42db-b6e9-53eed4316b01 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:47.724849 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57304df7-098e-42db-b6e9-53eed4316b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:47.742402 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-aebcf520-12dd-4070-b37d-5abcfbd3853e None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:08:47.744184 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-aebcf520-12dd-4070-b37d-5abcfbd3853e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:08:47.762102 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-57304df7-098e-42db-b6e9-53eed4316b01 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:08:47.763316 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-57304df7-098e-42db-b6e9-53eed4316b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:08:47.876369 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 48/100] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:08:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:08:47.877085 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:47.909949 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 53/101] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:08:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:08:47.911854 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57304df7-098e-42db-b6e9-53eed4316b01 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:47.943140 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc7529-d04b-4e5c-b059-8b0c77e4ba9a None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:47.943778 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc7529-d04b-4e5c-b059-8b0c77e4ba9a None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:47.944266 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc7529-d04b-4e5c-b059-8b0c77e4ba9a None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:47.946436 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 54/102] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:08:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:08:47.950353 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cc7529-d04b-4e5c-b059-8b0c77e4ba9a None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:48.039096 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57304df7-098e-42db-b6e9-53eed4316b01 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:48.080544 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e7d4b-b822-47ae-93ca-e9669c464791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:48.081383 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e7d4b-b822-47ae-93ca-e9669c464791 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:48.081660 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e7d4b-b822-47ae-93ca-e9669c464791 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:48.081933 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e7d4b-b822-47ae-93ca-e9669c464791 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:48.103486 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553e7d4b-b822-47ae-93ca-e9669c464791 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:08:48.119666 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-553e7d4b-b822-47ae-93ca-e9669c464791 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:08:48.119666 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:08:48.119666 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:08:48.119666 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:08:48.119666 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:08:48.119666 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:08:48.119666 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:08:48.119666 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:08:48.119666 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:08:48.119666 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:08:48.119666 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:08:48.119666 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:08:48.119666 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:08:48.119666 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:08:48.119666 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:08:48.119666 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:08:48.120580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:08:48.120580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:08:48.120580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:08:48.120580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:08:48.120580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:08:48.120580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:08:48.120580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:08:48.120580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:08:48.120580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:08:48.120580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:08:48.120580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:08:48.120580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:08:48.120580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:08:48.120580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:08:48.120580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:08:48.120580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:08:48.121390 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:08:48.121390 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:08:48.121390 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:08:48.121390 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:08:48.121390 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:08:48.121390 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:08:48.121390 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:08:48.121390 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:08:48.121390 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:08:48.121390 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:08:48.121390 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:08:48.121390 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:08:48.121390 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 49/103] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:08:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:08:48.122779 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b1655-aeb7-463f-b6dc-c8934d3be487 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:48.122779 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b1655-aeb7-463f-b6dc-c8934d3be487 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:48.122779 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b1655-aeb7-463f-b6dc-c8934d3be487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:48.141864 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-769b1655-aeb7-463f-b6dc-c8934d3be487 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:08:48.143042 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-769b1655-aeb7-463f-b6dc-c8934d3be487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:08:48.158788 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:48.175275 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cc7529-d04b-4e5c-b059-8b0c77e4ba9a None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:48.240138 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe8474-ae92-4a68-8214-175edc1c7a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:48.240881 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe8474-ae92-4a68-8214-175edc1c7a0a None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:48.241196 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe8474-ae92-4a68-8214-175edc1c7a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:48.241523 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe8474-ae92-4a68-8214-175edc1c7a0a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:48.256810 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe8474-ae92-4a68-8214-175edc1c7a0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:08:48.264139 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:48.268233 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-7fbe8474-ae92-4a68-8214-175edc1c7a0a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:08:48.268233 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:08:48.268233 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:08:48.268233 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:08:48.268233 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:08:48.268233 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:08:48.268233 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:08:48.268233 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:08:48.268233 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:08:48.268233 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:08:48.268233 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:08:48.268233 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:08:48.268233 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:08:48.268233 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:08:48.268233 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:08:48.268233 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:08:48.269277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:08:48.269277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:08:48.269277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:08:48.269277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:08:48.269277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:08:48.269277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:08:48.269277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:08:48.269277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:08:48.269277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:08:48.269277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:08:48.269277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:08:48.269277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:08:48.269277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:08:48.269277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:08:48.269277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:08:48.269277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:08:48.270231 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:08:48.270231 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:08:48.270231 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:08:48.270231 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:08:48.270231 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:08:48.270231 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:08:48.270231 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:08:48.270231 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:08:48.270231 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:08:48.270231 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:08:48.270231 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:08:48.270231 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:08:48.272348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 55/104] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 03:08:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:08:48.274379 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe8474-ae92-4a68-8214-175edc1c7a0a None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:48.298222 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 50/105] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:08:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:08:48.299347 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b1655-aeb7-463f-b6dc-c8934d3be487 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:48.406756 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b1655-aeb7-463f-b6dc-c8934d3be487 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:48.443540 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85c32d-f1a9-4333-afdc-25f63ea76010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:48.444801 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85c32d-f1a9-4333-afdc-25f63ea76010 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:48.445387 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85c32d-f1a9-4333-afdc-25f63ea76010 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:48.445816 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85c32d-f1a9-4333-afdc-25f63ea76010 None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:48.446855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a85c32d-f1a9-4333-afdc-25f63ea76010 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:08:48.447309 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a85c32d-f1a9-4333-afdc-25f63ea76010 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:48.449274 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a85c32d-f1a9-4333-afdc-25f63ea76010 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:48.459272 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 51/106] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:08:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:48.465074 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85c32d-f1a9-4333-afdc-25f63ea76010 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:48.532450 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe8474-ae92-4a68-8214-175edc1c7a0a None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:48.533624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:48.569312 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad048db-eae1-4d69-bfc3-7af0864afb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:48.570493 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad048db-eae1-4d69-bfc3-7af0864afb35 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:48.571124 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad048db-eae1-4d69-bfc3-7af0864afb35 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:48.571599 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad048db-eae1-4d69-bfc3-7af0864afb35 None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:48.572580 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad048db-eae1-4d69-bfc3-7af0864afb35 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:08:48.573114 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad048db-eae1-4d69-bfc3-7af0864afb35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:48.574783 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad048db-eae1-4d69-bfc3-7af0864afb35 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:48.583964 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 56/107] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 03:08:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:48.585154 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad048db-eae1-4d69-bfc3-7af0864afb35 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:48.597212 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85c32d-f1a9-4333-afdc-25f63ea76010 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:48.601794 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:48.637421 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69374abb-952f-4db6-9227-e353f0dd4992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:48.638264 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69374abb-952f-4db6-9227-e353f0dd4992 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:48.638548 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69374abb-952f-4db6-9227-e353f0dd4992 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:48.638829 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69374abb-952f-4db6-9227-e353f0dd4992 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:48.640119 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69374abb-952f-4db6-9227-e353f0dd4992 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:48.641733 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69374abb-952f-4db6-9227-e353f0dd4992 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:48.643371 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 52/108] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:08:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:48.649932 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b045c-9d7f-4c7d-b44d-5e2271f9fa9f None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:48.650432 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b045c-9d7f-4c7d-b44d-5e2271f9fa9f None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:48.650949 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b045c-9d7f-4c7d-b44d-5e2271f9fa9f None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:48.652548 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 53/109] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:08:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:08:48.653484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b045c-9d7f-4c7d-b44d-5e2271f9fa9f None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:48.695559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad048db-eae1-4d69-bfc3-7af0864afb35 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:48.731957 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0580557-4249-4d33-9f75-5236414068ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:48.733005 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0580557-4249-4d33-9f75-5236414068ea None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:48.733346 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0580557-4249-4d33-9f75-5236414068ea None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:48.733624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0580557-4249-4d33-9f75-5236414068ea None admin] PATH_INFO: `/v3/users/ba8ddcedde1a4bee9ba699929814f673` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:48.734817 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0580557-4249-4d33-9f75-5236414068ea None admin] RBAC: Authorizing `identity:get_user(user_id=ba8ddcedde1a4bee9ba699929814f673)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:48.736291 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0580557-4249-4d33-9f75-5236414068ea None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:48.742609 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 57/110] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:08:48 2022] GET /identity/v3/users/ba8ddcedde1a4bee9ba699929814f673 => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:48.742609 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0580557-4249-4d33-9f75-5236414068ea None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:48.771399 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b045c-9d7f-4c7d-b44d-5e2271f9fa9f None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:48.793320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:48.809529 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fa887c-cc68-4a76-8355-c5e47579e566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:48.810428 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa887c-cc68-4a76-8355-c5e47579e566 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:48.810714 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa887c-cc68-4a76-8355-c5e47579e566 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:48.810990 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa887c-cc68-4a76-8355-c5e47579e566 None admin] PATH_INFO: `/v3/users/3fbe4915d35149f6ab1a29cdfb821f19` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:48.825970 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fa887c-cc68-4a76-8355-c5e47579e566 None admin] RBAC: Authorizing `identity:get_user(user_id=3fbe4915d35149f6ab1a29cdfb821f19)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:48.827455 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fa887c-cc68-4a76-8355-c5e47579e566 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:48.828107 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:48.829469 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 54/111] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:08:48 2022] GET /identity/v3/users/3fbe4915d35149f6ab1a29cdfb821f19 => generated 289 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:48.830560 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0fbd60-7e35-4466-840d-3ff996660b29 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:48.830850 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0fbd60-7e35-4466-840d-3ff996660b29 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:48.831130 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0fbd60-7e35-4466-840d-3ff996660b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:48.849656 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-fb0fbd60-7e35-4466-840d-3ff996660b29 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:08:48.851021 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-fb0fbd60-7e35-4466-840d-3ff996660b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:08:48.852882 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0580557-4249-4d33-9f75-5236414068ea None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:48.890068 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c53e9-cd3a-4560-a368-1166ae11d28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:48.890742 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c53e9-cd3a-4560-a368-1166ae11d28e None admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:48.891027 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c53e9-cd3a-4560-a368-1166ae11d28e None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:48.891303 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5c53e9-cd3a-4560-a368-1166ae11d28e None admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:48.892025 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c53e9-cd3a-4560-a368-1166ae11d28e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:48.893623 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5c53e9-cd3a-4560-a368-1166ae11d28e None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:48.923767 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 55/112] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:08:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:08:48.925741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312617c9-b837-4fa4-8378-9efcf2458846 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:48.926238 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312617c9-b837-4fa4-8378-9efcf2458846 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:48.926709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312617c9-b837-4fa4-8378-9efcf2458846 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:48.928417 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 56/113] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:08:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:08:48.934352 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312617c9-b837-4fa4-8378-9efcf2458846 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:48.941399 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 58/114] 10.222.0.50 () {60 vars in 1141 bytes} [Fri May 27 03:08:48 2022] POST /identity/v3/projects => generated 302 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:08:48.942778 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c53e9-cd3a-4560-a368-1166ae11d28e None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:49.069192 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312617c9-b837-4fa4-8378-9efcf2458846 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:49.080965 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:49.088312 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5c53e9-cd3a-4560-a368-1166ae11d28e None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:49.109952 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc86cd6d-bcec-4516-bf26-81b6fb80e6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:49.110893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86cd6d-bcec-4516-bf26-81b6fb80e6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:49.111181 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86cd6d-bcec-4516-bf26-81b6fb80e6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:49.111457 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc86cd6d-bcec-4516-bf26-81b6fb80e6f2 None admin] PATH_INFO: `/v3/projects/4edef8e255584377bf94a9ac9712b4b7` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:49.112704 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc86cd6d-bcec-4516-bf26-81b6fb80e6f2 None admin] RBAC: Authorizing `identity:get_project(project_id=4edef8e255584377bf94a9ac9712b4b7)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:49.114258 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc86cd6d-bcec-4516-bf26-81b6fb80e6f2 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:49.116170 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 57/115] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:08:48 2022] GET /identity/v3/projects/4edef8e255584377bf94a9ac9712b4b7 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:49.117732 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b855c7-9fc4-4925-be28-7f71cc969cd6 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:49.118018 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b855c7-9fc4-4925-be28-7f71cc969cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:49.118296 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b855c7-9fc4-4925-be28-7f71cc969cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:49.128828 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:49.136542 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-f0b855c7-9fc4-4925-be28-7f71cc969cd6 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:08:49.137861 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-f0b855c7-9fc4-4925-be28-7f71cc969cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:08:49.138109 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0dbe16-5b8e-4668-a1a0-045df3e5b76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:49.139109 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0dbe16-5b8e-4668-a1a0-045df3e5b76f None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:49.139637 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0dbe16-5b8e-4668-a1a0-045df3e5b76f None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:49.140075 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0dbe16-5b8e-4668-a1a0-045df3e5b76f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:49.209568 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 58/116] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:08:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:08:49.210503 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b855c7-9fc4-4925-be28-7f71cc969cd6 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:49.221934 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0dbe16-5b8e-4668-a1a0-045df3e5b76f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:49.224080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0dbe16-5b8e-4668-a1a0-045df3e5b76f None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:49.314191 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b855c7-9fc4-4925-be28-7f71cc969cd6 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:49.323184 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:49.337081 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 59/117] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:08:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:08:49.338487 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7a2c4-6f09-40b4-85ba-58500ebcc39f None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:49.338779 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7a2c4-6f09-40b4-85ba-58500ebcc39f None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:49.339054 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7a2c4-6f09-40b4-85ba-58500ebcc39f None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:49.344452 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 60/118] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:08:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:08:49.345331 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7a2c4-6f09-40b4-85ba-58500ebcc39f None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:49.357647 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89171d98-0a20-41af-8349-045e66140fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:49.358775 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89171d98-0a20-41af-8349-045e66140fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:49.359339 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89171d98-0a20-41af-8349-045e66140fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:49.359654 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89171d98-0a20-41af-8349-045e66140fc7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:49.371455 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89171d98-0a20-41af-8349-045e66140fc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:08:49.377661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:49.379275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-89171d98-0a20-41af-8349-045e66140fc7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:08:49.379275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:08:49.379275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:08:49.379275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:08:49.379275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:08:49.379275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:08:49.379275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:08:49.379275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:08:49.379275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:08:49.379275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:08:49.379275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:08:49.379275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:08:49.379275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:08:49.379275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:08:49.379275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:08:49.379275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:08:49.380559 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:08:49.380559 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:08:49.380559 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:08:49.380559 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:08:49.380559 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:08:49.380559 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:08:49.380559 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:08:49.380559 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:08:49.380559 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:08:49.380559 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:08:49.380559 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:08:49.380559 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:08:49.380559 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:08:49.380559 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:08:49.380559 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:08:49.380559 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:08:49.381408 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:08:49.381408 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:08:49.381408 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:08:49.381408 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:08:49.381408 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:08:49.381408 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:08:49.381408 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:08:49.381408 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:08:49.381408 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:08:49.381408 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:08:49.381408 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:08:49.381408 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:08:49.381408 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 59/119] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:08:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:08:49.381408 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89171d98-0a20-41af-8349-045e66140fc7 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:49.460489 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7a2c4-6f09-40b4-85ba-58500ebcc39f None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:49.488624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89171d98-0a20-41af-8349-045e66140fc7 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:49.503830 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c03843-e61e-49d5-b31b-fc22da68684b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:49.505253 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c03843-e61e-49d5-b31b-fc22da68684b None admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:49.505548 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c03843-e61e-49d5-b31b-fc22da68684b None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:49.505812 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c03843-e61e-49d5-b31b-fc22da68684b None admin] PATH_INFO: `/v3/projects/4edef8e255584377bf94a9ac9712b4b7/users/ba8ddcedde1a4bee9ba699929814f673/roles/9e63b73b3ce24bb3ad214ecdb6132696` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:49.512782 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c03843-e61e-49d5-b31b-fc22da68684b None admin] RBAC: Authorizing `identity:create_grant(project_id=4edef8e255584377bf94a9ac9712b4b7, user_id=ba8ddcedde1a4bee9ba699929814f673, role_id=9e63b73b3ce24bb3ad214ecdb6132696)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:49.514319 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c03843-e61e-49d5-b31b-fc22da68684b None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:49.526287 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318963aa-c4d8-495f-b47e-c4c4e2fa8c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:49.527104 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318963aa-c4d8-495f-b47e-c4c4e2fa8c17 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:49.527389 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318963aa-c4d8-495f-b47e-c4c4e2fa8c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:49.527759 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318963aa-c4d8-495f-b47e-c4c4e2fa8c17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:49.531266 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 61/120] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 03:08:49 2022] PUT /identity/v3/projects/4edef8e255584377bf94a9ac9712b4b7/users/ba8ddcedde1a4bee9ba699929814f673/roles/9e63b73b3ce24bb3ad214ecdb6132696 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:08:49.532089 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c03843-e61e-49d5-b31b-fc22da68684b None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:49.540339 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318963aa-c4d8-495f-b47e-c4c4e2fa8c17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:08:49.548219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-318963aa-c4d8-495f-b47e-c4c4e2fa8c17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:08:49.548219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:08:49.548219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:08:49.548219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:08:49.548219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:08:49.548219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:08:49.548219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:08:49.548219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:08:49.548219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:08:49.548219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:08:49.548219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:08:49.548219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:08:49.548219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:08:49.548219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:08:49.548219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:08:49.548219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:08:49.549206 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:08:49.549206 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:08:49.549206 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:08:49.549206 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:08:49.549206 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:08:49.549206 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:08:49.549206 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:08:49.549206 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:08:49.549206 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:08:49.549206 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:08:49.549206 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:08:49.549206 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:08:49.549206 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:08:49.549206 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:08:49.549206 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:08:49.549206 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:08:49.550004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:08:49.550004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:08:49.550004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:08:49.550004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:08:49.550004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:08:49.550004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:08:49.550004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:08:49.550004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:08:49.550004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:08:49.550004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:08:49.550004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:08:49.550004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:08:49.550004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 60/121] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 03:08:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:08:49.551291 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27ca39-58fe-4128-b353-643e02a84210 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:49.551291 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27ca39-58fe-4128-b353-643e02a84210 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:49.551459 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27ca39-58fe-4128-b353-643e02a84210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:49.570506 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-8b27ca39-58fe-4128-b353-643e02a84210 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:08:49.572175 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-8b27ca39-58fe-4128-b353-643e02a84210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:08:49.623088 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:49.665504 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 61/122] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:08:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:08:49.667013 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27ca39-58fe-4128-b353-643e02a84210 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:49.668381 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:49.680094 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c03843-e61e-49d5-b31b-fc22da68684b None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:49.720064 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a31291e-6395-4fac-bc42-3b83fda7337e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:49.721021 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31291e-6395-4fac-bc42-3b83fda7337e None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:49.721360 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31291e-6395-4fac-bc42-3b83fda7337e None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:49.721640 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31291e-6395-4fac-bc42-3b83fda7337e None admin] PATH_INFO: `/v3/projects/4edef8e255584377bf94a9ac9712b4b7` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:49.722840 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a31291e-6395-4fac-bc42-3b83fda7337e None admin] RBAC: Authorizing `identity:get_project(project_id=4edef8e255584377bf94a9ac9712b4b7)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:49.724337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a31291e-6395-4fac-bc42-3b83fda7337e None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:49.726150 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 62/123] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:08:49 2022] GET /identity/v3/projects/4edef8e255584377bf94a9ac9712b4b7 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:49.726937 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a31291e-6395-4fac-bc42-3b83fda7337e None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:49.822517 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27ca39-58fe-4128-b353-643e02a84210 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:49.859667 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b83bf-9bea-4b46-a287-1cb1e69005a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:49.860350 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b83bf-9bea-4b46-a287-1cb1e69005a8 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:49.860635 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b83bf-9bea-4b46-a287-1cb1e69005a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:49.860911 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b83bf-9bea-4b46-a287-1cb1e69005a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:49.861715 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b83bf-9bea-4b46-a287-1cb1e69005a8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:08:49.862020 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b83bf-9bea-4b46-a287-1cb1e69005a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:49.862246 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a31291e-6395-4fac-bc42-3b83fda7337e None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:49.863449 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b83bf-9bea-4b46-a287-1cb1e69005a8 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:49.872757 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 62/124] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 03:08:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:49.878376 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b83bf-9bea-4b46-a287-1cb1e69005a8 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:49.898876 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:49.906049 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6f58f-9abc-43b7-940f-26e05e4669fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:49.906751 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6f58f-9abc-43b7-940f-26e05e4669fe None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:49.907028 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6f58f-9abc-43b7-940f-26e05e4669fe None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:49.907322 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6f58f-9abc-43b7-940f-26e05e4669fe None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:49.908231 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d6f58f-9abc-43b7-940f-26e05e4669fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:08:49.908918 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d6f58f-9abc-43b7-940f-26e05e4669fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:49.910601 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d6f58f-9abc-43b7-940f-26e05e4669fe None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:49.922638 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 63/125] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:08:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:49.923489 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6f58f-9abc-43b7-940f-26e05e4669fe None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:49.962012 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:50.006010 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b83bf-9bea-4b46-a287-1cb1e69005a8 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:50.049172 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ed96e-096d-48bd-9652-db5cb4ef24b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:50.050472 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ed96e-096d-48bd-9652-db5cb4ef24b9 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:50.051117 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ed96e-096d-48bd-9652-db5cb4ef24b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:50.051546 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ed96e-096d-48bd-9652-db5cb4ef24b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:50.059038 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6f58f-9abc-43b7-940f-26e05e4669fe None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:50.065301 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ed96e-096d-48bd-9652-db5cb4ef24b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:08:50.075409 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-5d7ed96e-096d-48bd-9652-db5cb4ef24b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:08:50.075409 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:08:50.075409 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:08:50.075409 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:08:50.075409 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:08:50.075409 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:08:50.075409 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:08:50.075409 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:08:50.075409 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:08:50.075409 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:08:50.075409 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:08:50.075409 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:08:50.075409 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:08:50.075409 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:08:50.075409 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:08:50.075409 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:08:50.076336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:08:50.076336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:08:50.076336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:08:50.076336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:08:50.076336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:08:50.076336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:08:50.076336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:08:50.076336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:08:50.076336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:08:50.076336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:08:50.076336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:08:50.076336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:08:50.076336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:08:50.076336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:08:50.076336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:08:50.076336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:08:50.077126 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:08:50.077126 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:08:50.077126 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:08:50.077126 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:08:50.077126 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:08:50.077126 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:08:50.077126 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:08:50.077126 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:08:50.077126 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:08:50.077126 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:08:50.077126 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:08:50.077126 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:08:50.078893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 63/126] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:08:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:08:50.081384 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ed96e-096d-48bd-9652-db5cb4ef24b9 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:50.102792 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957d4b3-cf8c-42c7-abe5-2550e48ecec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:50.103928 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957d4b3-cf8c-42c7-abe5-2550e48ecec9 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:50.104220 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957d4b3-cf8c-42c7-abe5-2550e48ecec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:50.104499 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957d4b3-cf8c-42c7-abe5-2550e48ecec9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:50.105652 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3957d4b3-cf8c-42c7-abe5-2550e48ecec9 None admin] RBAC: Adding query filter params (role.id=6479235705f74061aff23df5c9f89e33, scope.project.id=4edef8e255584377bf94a9ac9712b4b7, user.id=3fbe4915d35149f6ab1a29cdfb821f19) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:08:50.105973 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3957d4b3-cf8c-42c7-abe5-2550e48ecec9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:50.107706 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3957d4b3-cf8c-42c7-abe5-2550e48ecec9 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:50.116349 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 64/127] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:08:49 2022] GET /identity/v3/role_assignments?user.id=3fbe4915d35149f6ab1a29cdfb821f19&scope.project.id=4edef8e255584377bf94a9ac9712b4b7&role.id=6479235705f74061aff23df5c9f89e33 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:50.117814 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957d4b3-cf8c-42c7-abe5-2550e48ecec9 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:50.194063 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:50.220193 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ed96e-096d-48bd-9652-db5cb4ef24b9 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:50.223697 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:50.243593 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957d4b3-cf8c-42c7-abe5-2550e48ecec9 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:50.263086 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d864236e-9762-4d92-a298-b840e08135d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:50.264054 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864236e-9762-4d92-a298-b840e08135d8 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:50.264340 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864236e-9762-4d92-a298-b840e08135d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:50.264665 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864236e-9762-4d92-a298-b840e08135d8 None admin] PATH_INFO: `/v3/users/ba8ddcedde1a4bee9ba699929814f673` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:50.265925 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d864236e-9762-4d92-a298-b840e08135d8 None admin] RBAC: Authorizing `identity:get_user(user_id=ba8ddcedde1a4bee9ba699929814f673)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:50.268065 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d864236e-9762-4d92-a298-b840e08135d8 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:50.270207 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 64/128] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:08:50 2022] GET /identity/v3/users/ba8ddcedde1a4bee9ba699929814f673 => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:50.271176 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d864236e-9762-4d92-a298-b840e08135d8 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:50.291409 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a85192-c021-41d8-b140-9a0b8a87a14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:50.292685 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a85192-c021-41d8-b140-9a0b8a87a14b None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:50.293206 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a85192-c021-41d8-b140-9a0b8a87a14b None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:50.293699 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a85192-c021-41d8-b140-9a0b8a87a14b None admin] PATH_INFO: `/v3/users/4a96d7acf93e46c7ac5700560db36dbe` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:50.295105 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a85192-c021-41d8-b140-9a0b8a87a14b None admin] RBAC: Authorizing `identity:get_user(user_id=4a96d7acf93e46c7ac5700560db36dbe)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:50.296874 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a85192-c021-41d8-b140-9a0b8a87a14b None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:50.299409 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 65/129] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:08:50 2022] GET /identity/v3/users/4a96d7acf93e46c7ac5700560db36dbe => generated 247 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:50.300622 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a85192-c021-41d8-b140-9a0b8a87a14b None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:50.394650 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d864236e-9762-4d92-a298-b840e08135d8 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:50.412326 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a85192-c021-41d8-b140-9a0b8a87a14b None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:50.433912 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cc35a9-b038-463d-ac8f-1ee8a9f36ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:50.434641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc35a9-b038-463d-ac8f-1ee8a9f36ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:50.434953 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc35a9-b038-463d-ac8f-1ee8a9f36ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:50.435230 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc35a9-b038-463d-ac8f-1ee8a9f36ec0 None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:50.435992 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cc35a9-b038-463d-ac8f-1ee8a9f36ec0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:08:50.436302 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cc35a9-b038-463d-ac8f-1ee8a9f36ec0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:50.437860 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cc35a9-b038-463d-ac8f-1ee8a9f36ec0 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:50.447562 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 65/130] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 03:08:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:50.448479 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cc35a9-b038-463d-ac8f-1ee8a9f36ec0 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:50.457311 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b9b8e7-f6f1-4a05-8fc2-ac0ca888d9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:50.458565 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9b8e7-f6f1-4a05-8fc2-ac0ca888d9a2 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:50.459044 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9b8e7-f6f1-4a05-8fc2-ac0ca888d9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:50.459542 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9b8e7-f6f1-4a05-8fc2-ac0ca888d9a2 None admin] PATH_INFO: `/v3/projects/4edef8e255584377bf94a9ac9712b4b7` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:50.460108 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:50.466254 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b9b8e7-f6f1-4a05-8fc2-ac0ca888d9a2 None admin] RBAC: Authorizing `identity:get_project(project_id=4edef8e255584377bf94a9ac9712b4b7)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:50.467471 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b9b8e7-f6f1-4a05-8fc2-ac0ca888d9a2 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:50.469317 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 66/131] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:08:50 2022] GET /identity/v3/projects/4edef8e255584377bf94a9ac9712b4b7 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:50.470100 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b9b8e7-f6f1-4a05-8fc2-ac0ca888d9a2 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:50.511977 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:50.567244 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cc35a9-b038-463d-ac8f-1ee8a9f36ec0 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:50.605146 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f43e20-9f3c-4637-a765-91089fbece6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:50.606524 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f43e20-9f3c-4637-a765-91089fbece6d None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:50.607011 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f43e20-9f3c-4637-a765-91089fbece6d None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:50.607508 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f43e20-9f3c-4637-a765-91089fbece6d None admin] PATH_INFO: `/v3/projects/4edef8e255584377bf94a9ac9712b4b7` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:50.609234 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f43e20-9f3c-4637-a765-91089fbece6d None admin] RBAC: Authorizing `identity:get_project(project_id=4edef8e255584377bf94a9ac9712b4b7)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:50.611052 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f43e20-9f3c-4637-a765-91089fbece6d None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:50.612990 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 66/132] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:08:50 2022] GET /identity/v3/projects/4edef8e255584377bf94a9ac9712b4b7 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:50.614081 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f43e20-9f3c-4637-a765-91089fbece6d None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:50.697087 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b9b8e7-f6f1-4a05-8fc2-ac0ca888d9a2 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:50.729412 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:50.762711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f43e20-9f3c-4637-a765-91089fbece6d None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:50.765902 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82131769-1c65-4181-be0f-ab843f3dd727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:50.766806 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82131769-1c65-4181-be0f-ab843f3dd727 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:50.767087 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82131769-1c65-4181-be0f-ab843f3dd727 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:50.767356 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82131769-1c65-4181-be0f-ab843f3dd727 None admin] PATH_INFO: `/v3/users/ba8ddcedde1a4bee9ba699929814f673` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:50.771630 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82131769-1c65-4181-be0f-ab843f3dd727 None admin] RBAC: Authorizing `identity:get_user(user_id=ba8ddcedde1a4bee9ba699929814f673)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:50.773367 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82131769-1c65-4181-be0f-ab843f3dd727 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:50.776542 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 67/133] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:08:50 2022] GET /identity/v3/users/ba8ddcedde1a4bee9ba699929814f673 => generated 275 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:50.781979 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82131769-1c65-4181-be0f-ab843f3dd727 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:50.828025 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3787dd8d-150a-44ad-b7bd-83f3ec03d63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:50.828990 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3787dd8d-150a-44ad-b7bd-83f3ec03d63e None admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:50.829278 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3787dd8d-150a-44ad-b7bd-83f3ec03d63e None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:50.829553 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3787dd8d-150a-44ad-b7bd-83f3ec03d63e None admin] PATH_INFO: `/v3/projects/4edef8e255584377bf94a9ac9712b4b7/users/ba8ddcedde1a4bee9ba699929814f673/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:50.841825 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3787dd8d-150a-44ad-b7bd-83f3ec03d63e None admin] RBAC: Authorizing `identity:create_grant(project_id=4edef8e255584377bf94a9ac9712b4b7, user_id=ba8ddcedde1a4bee9ba699929814f673, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:50.843320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3787dd8d-150a-44ad-b7bd-83f3ec03d63e None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:50.863795 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 67/134] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 03:08:50 2022] PUT /identity/v3/projects/4edef8e255584377bf94a9ac9712b4b7/users/ba8ddcedde1a4bee9ba699929814f673/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:08:50.865040 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3787dd8d-150a-44ad-b7bd-83f3ec03d63e None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:50.961386 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:50.983597 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82131769-1c65-4181-be0f-ab843f3dd727 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:50.996128 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3787dd8d-150a-44ad-b7bd-83f3ec03d63e None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:51.060133 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320816af-d41c-4a4e-ab91-473fc54e3356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:51.061071 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320816af-d41c-4a4e-ab91-473fc54e3356 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:51.061356 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320816af-d41c-4a4e-ab91-473fc54e3356 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:51.061633 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320816af-d41c-4a4e-ab91-473fc54e3356 None admin] PATH_INFO: `/v3/projects/0f4c723f76b04fab9f9ac3617cf3fb8b` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:51.065999 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320816af-d41c-4a4e-ab91-473fc54e3356 None admin] RBAC: Authorizing `identity:get_project(project_id=0f4c723f76b04fab9f9ac3617cf3fb8b)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:51.067738 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320816af-d41c-4a4e-ab91-473fc54e3356 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:51.069741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 68/135] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:08:50 2022] GET /identity/v3/projects/0f4c723f76b04fab9f9ac3617cf3fb8b => generated 312 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:51.077442 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320816af-d41c-4a4e-ab91-473fc54e3356 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:51.082768 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:51.083758 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84cca9d-6ba9-41c1-b501-5366ae1f2050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:51.084684 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84cca9d-6ba9-41c1-b501-5366ae1f2050 None admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:51.084999 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84cca9d-6ba9-41c1-b501-5366ae1f2050 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:51.085304 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84cca9d-6ba9-41c1-b501-5366ae1f2050 None admin] PATH_INFO: `/v3/projects/4edef8e255584377bf94a9ac9712b4b7/users/4a96d7acf93e46c7ac5700560db36dbe/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:51.090780 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84cca9d-6ba9-41c1-b501-5366ae1f2050 None admin] RBAC: Authorizing `identity:create_grant(project_id=4edef8e255584377bf94a9ac9712b4b7, user_id=4a96d7acf93e46c7ac5700560db36dbe, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:51.092285 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84cca9d-6ba9-41c1-b501-5366ae1f2050 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:51.120565 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 68/136] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 03:08:50 2022] PUT /identity/v3/projects/4edef8e255584377bf94a9ac9712b4b7/users/4a96d7acf93e46c7ac5700560db36dbe/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:08:51.214760 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320816af-d41c-4a4e-ab91-473fc54e3356 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:51.255623 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d65cc-b385-48b3-9f43-1e84165e845e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:51.257080 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d65cc-b385-48b3-9f43-1e84165e845e None admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:51.257756 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d65cc-b385-48b3-9f43-1e84165e845e None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:51.258442 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d65cc-b385-48b3-9f43-1e84165e845e None admin] PATH_INFO: `/v3/projects/0f4c723f76b04fab9f9ac3617cf3fb8b/users/ba8ddcedde1a4bee9ba699929814f673/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:51.261142 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10d65cc-b385-48b3-9f43-1e84165e845e None admin] RBAC: Authorizing `identity:create_grant(project_id=0f4c723f76b04fab9f9ac3617cf3fb8b, user_id=ba8ddcedde1a4bee9ba699929814f673, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:51.262963 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10d65cc-b385-48b3-9f43-1e84165e845e None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:51.283136 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 69/137] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 03:08:51 2022] PUT /identity/v3/projects/0f4c723f76b04fab9f9ac3617cf3fb8b/users/ba8ddcedde1a4bee9ba699929814f673/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:08:51.798023 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f63127-d1c1-4e20-a3d5-a3704f687aca None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:51.798731 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f63127-d1c1-4e20-a3d5-a3704f687aca None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:51.799211 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f63127-d1c1-4e20-a3d5-a3704f687aca None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:51.800818 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 69/138] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:08:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:08:51.806490 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d862e1d-33d3-463c-b063-db2188a22f11 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:51.807050 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d862e1d-33d3-463c-b063-db2188a22f11 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:51.807516 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d862e1d-33d3-463c-b063-db2188a22f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:51.830502 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-7d862e1d-33d3-463c-b063-db2188a22f11 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:08:51.832483 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-7d862e1d-33d3-463c-b063-db2188a22f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:08:51.922754 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:51.936327 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 70/139] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:08:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:08:51.972925 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f63127-d1c1-4e20-a3d5-a3704f687aca None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:52.114712 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f63127-d1c1-4e20-a3d5-a3704f687aca None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:52.157615 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edbbc22-8750-48b5-b381-e4b98ecfc8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:52.158434 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edbbc22-8750-48b5-b381-e4b98ecfc8ed None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:52.158718 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edbbc22-8750-48b5-b381-e4b98ecfc8ed None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:52.158991 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edbbc22-8750-48b5-b381-e4b98ecfc8ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:52.165684 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edbbc22-8750-48b5-b381-e4b98ecfc8ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:52.167191 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edbbc22-8750-48b5-b381-e4b98ecfc8ed None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:52.168862 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 70/140] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:08:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:52.174608 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d862e1d-33d3-463c-b063-db2188a22f11 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:52.309646 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d862e1d-33d3-463c-b063-db2188a22f11 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:52.365641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac6458-fce5-4cfd-8ade-d5144dc2d45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:52.366839 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac6458-fce5-4cfd-8ade-d5144dc2d45e None admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:52.367565 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac6458-fce5-4cfd-8ade-d5144dc2d45e None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:52.368072 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac6458-fce5-4cfd-8ade-d5144dc2d45e None admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:52.368983 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ac6458-fce5-4cfd-8ade-d5144dc2d45e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:52.370679 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ac6458-fce5-4cfd-8ade-d5144dc2d45e None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:52.394039 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 71/141] 10.222.0.50 () {60 vars in 1132 bytes} [Fri May 27 03:08:52 2022] POST /identity/v3/users => generated 283 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:08:52.675164 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec93fa5-2114-4a21-8251-8da5079949e4 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:52.675844 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec93fa5-2114-4a21-8251-8da5079949e4 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:52.676327 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec93fa5-2114-4a21-8251-8da5079949e4 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:52.678206 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 71/142] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:08:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:08:52.684271 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cbee15-f215-4e24-b052-2f2fb8639c11 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:52.684899 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cbee15-f215-4e24-b052-2f2fb8639c11 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:52.685398 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cbee15-f215-4e24-b052-2f2fb8639c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:52.704511 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-b4cbee15-f215-4e24-b052-2f2fb8639c11 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:08:52.710302 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-b4cbee15-f215-4e24-b052-2f2fb8639c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:08:52.774862 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:52.801083 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 72/143] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:08:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:08:52.836853 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec93fa5-2114-4a21-8251-8da5079949e4 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:52.870313 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:52.974083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec93fa5-2114-4a21-8251-8da5079949e4 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:53.017794 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eded83-2b95-4d95-a26f-3058ca588999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:53.018653 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eded83-2b95-4d95-a26f-3058ca588999 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:53.018933 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eded83-2b95-4d95-a26f-3058ca588999 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:53.019202 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eded83-2b95-4d95-a26f-3058ca588999 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:53.034327 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eded83-2b95-4d95-a26f-3058ca588999 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:08:53.043290 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-80eded83-2b95-4d95-a26f-3058ca588999 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:08:53.043290 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:08:53.043290 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:08:53.043290 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:08:53.043290 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:08:53.043290 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:08:53.043290 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:08:53.043290 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:08:53.043290 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:08:53.043290 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:08:53.043290 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:08:53.043290 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:08:53.043290 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:08:53.043290 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:08:53.043290 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:08:53.043290 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:08:53.044188 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:08:53.044188 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:08:53.044188 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:08:53.044188 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:08:53.044188 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:08:53.044188 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:08:53.044188 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:08:53.044188 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:08:53.044188 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:08:53.044188 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:08:53.044188 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:08:53.044188 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:08:53.044188 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:08:53.044188 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:08:53.044188 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:08:53.044188 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:08:53.044998 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:08:53.044998 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:08:53.044998 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:08:53.044998 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:08:53.044998 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:08:53.044998 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:08:53.044998 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:08:53.044998 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:08:53.044998 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:08:53.044998 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:08:53.044998 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:08:53.044998 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:08:53.044998 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 72/144] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 03:08:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:08:53.050975 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cbee15-f215-4e24-b052-2f2fb8639c11 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:53.200165 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cbee15-f215-4e24-b052-2f2fb8639c11 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:53.240935 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a83f068-febe-4a40-a16d-09e887d3989a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:53.242369 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83f068-febe-4a40-a16d-09e887d3989a None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:53.242926 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83f068-febe-4a40-a16d-09e887d3989a None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:53.243392 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83f068-febe-4a40-a16d-09e887d3989a None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:53.244336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a83f068-febe-4a40-a16d-09e887d3989a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:08:53.244847 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a83f068-febe-4a40-a16d-09e887d3989a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:53.246480 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a83f068-febe-4a40-a16d-09e887d3989a None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:53.255723 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 73/145] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:08:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:53.262097 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eded83-2b95-4d95-a26f-3058ca588999 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:53.267820 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9388ee-c50a-474d-9e37-8e73dc3c06a8 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:53.268365 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9388ee-c50a-474d-9e37-8e73dc3c06a8 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:53.268854 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9388ee-c50a-474d-9e37-8e73dc3c06a8 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:53.270767 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 74/146] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:08:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:08:53.280617 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa21c42-fc03-4a27-9ea9-eeef97cff873 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:53.281307 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa21c42-fc03-4a27-9ea9-eeef97cff873 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:53.281855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa21c42-fc03-4a27-9ea9-eeef97cff873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:53.304851 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-3fa21c42-fc03-4a27-9ea9-eeef97cff873 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:08:53.306272 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-3fa21c42-fc03-4a27-9ea9-eeef97cff873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:08:53.352741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:53.393884 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 75/147] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:08:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:08:53.417622 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa21c42-fc03-4a27-9ea9-eeef97cff873 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:53.435983 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:53.465271 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eded83-2b95-4d95-a26f-3058ca588999 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:53.521795 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58f1ec1-d94a-4ff5-af8a-4f2143ab4c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:53.523094 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f1ec1-d94a-4ff5-af8a-4f2143ab4c97 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:53.523546 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f1ec1-d94a-4ff5-af8a-4f2143ab4c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:53.524040 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f1ec1-d94a-4ff5-af8a-4f2143ab4c97 None admin] PATH_INFO: `/v3/users/ba8ddcedde1a4bee9ba699929814f673` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:53.525519 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58f1ec1-d94a-4ff5-af8a-4f2143ab4c97 None admin] RBAC: Authorizing `identity:get_user(user_id=ba8ddcedde1a4bee9ba699929814f673)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:53.527152 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58f1ec1-d94a-4ff5-af8a-4f2143ab4c97 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:53.528973 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 73/148] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:08:53 2022] GET /identity/v3/users/ba8ddcedde1a4bee9ba699929814f673 => generated 275 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:53.531989 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa21c42-fc03-4a27-9ea9-eeef97cff873 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:53.534673 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58f1ec1-d94a-4ff5-af8a-4f2143ab4c97 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:53.570197 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcceb1a1-978f-44ce-88eb-16fd716a3b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:53.571402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcceb1a1-978f-44ce-88eb-16fd716a3b27 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:53.571574 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcceb1a1-978f-44ce-88eb-16fd716a3b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:53.573151 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcceb1a1-978f-44ce-88eb-16fd716a3b27 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:53.584823 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcceb1a1-978f-44ce-88eb-16fd716a3b27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:08:53.593429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-fcceb1a1-978f-44ce-88eb-16fd716a3b27 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:08:53.593429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:08:53.593429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:08:53.593429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:08:53.593429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:08:53.593429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:08:53.593429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:08:53.593429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:08:53.593429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:08:53.593429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:08:53.593429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:08:53.593429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:08:53.593429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:08:53.593429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:08:53.593429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:08:53.593429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:08:53.594280 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:08:53.594280 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:08:53.594280 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:08:53.594280 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:08:53.594280 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:08:53.594280 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:08:53.594280 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:08:53.594280 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:08:53.594280 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:08:53.594280 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:08:53.594280 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:08:53.594280 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:08:53.594280 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:08:53.594280 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:08:53.594280 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:08:53.594280 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:08:53.595009 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:08:53.595009 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:08:53.595009 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:08:53.595009 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:08:53.595009 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:08:53.595009 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:08:53.595009 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:08:53.595009 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:08:53.595009 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:08:53.595009 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:08:53.595009 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:08:53.595009 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:08:53.595009 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 76/149] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:08:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:08:53.601079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcceb1a1-978f-44ce-88eb-16fd716a3b27 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:53.637982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:53.648185 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58f1ec1-d94a-4ff5-af8a-4f2143ab4c97 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:53.686418 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4400adc2-cd27-4761-ac08-2bc4659db0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:53.687450 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4400adc2-cd27-4761-ac08-2bc4659db0d6 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:53.687922 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4400adc2-cd27-4761-ac08-2bc4659db0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:53.688384 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4400adc2-cd27-4761-ac08-2bc4659db0d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:53.704622 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:53.722089 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcceb1a1-978f-44ce-88eb-16fd716a3b27 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:53.762363 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a8f7ae-e3e6-403d-9822-e2bd69b8ad80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:53.763037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4400adc2-cd27-4761-ac08-2bc4659db0d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:53.763461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8f7ae-e3e6-403d-9822-e2bd69b8ad80 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:53.763881 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8f7ae-e3e6-403d-9822-e2bd69b8ad80 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:53.764317 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a8f7ae-e3e6-403d-9822-e2bd69b8ad80 None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:53.764929 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4400adc2-cd27-4761-ac08-2bc4659db0d6 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:53.765427 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a8f7ae-e3e6-403d-9822-e2bd69b8ad80 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:08:53.765870 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a8f7ae-e3e6-403d-9822-e2bd69b8ad80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:53.767570 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a8f7ae-e3e6-403d-9822-e2bd69b8ad80 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:53.777904 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 77/150] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 03:08:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:53.784292 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a8f7ae-e3e6-403d-9822-e2bd69b8ad80 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:53.949330 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:53.952089 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 74/151] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:08:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:08:53.958009 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4400adc2-cd27-4761-ac08-2bc4659db0d6 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:53.964453 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a8f7ae-e3e6-403d-9822-e2bd69b8ad80 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:54.014872 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:54.027262 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6baba44-ac55-4717-ad45-689a95080fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:54.028148 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6baba44-ac55-4717-ad45-689a95080fff None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:54.028368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6baba44-ac55-4717-ad45-689a95080fff None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:54.028632 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6baba44-ac55-4717-ad45-689a95080fff None admin] PATH_INFO: `/v3/users/3fbe4915d35149f6ab1a29cdfb821f19` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:54.030025 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6baba44-ac55-4717-ad45-689a95080fff None admin] RBAC: Authorizing `identity:get_user(user_id=3fbe4915d35149f6ab1a29cdfb821f19)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:54.031463 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6baba44-ac55-4717-ad45-689a95080fff None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:54.033563 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 78/152] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:08:53 2022] GET /identity/v3/users/3fbe4915d35149f6ab1a29cdfb821f19 => generated 289 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:54.039614 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6baba44-ac55-4717-ad45-689a95080fff None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:54.075989 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4400adc2-cd27-4761-ac08-2bc4659db0d6 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:54.115723 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aafac6-301e-44ba-823a-63202a2077b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:54.116820 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aafac6-301e-44ba-823a-63202a2077b0 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:54.117355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aafac6-301e-44ba-823a-63202a2077b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:54.117797 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aafac6-301e-44ba-823a-63202a2077b0 None admin] PATH_INFO: `/v3/projects/4edef8e255584377bf94a9ac9712b4b7` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:54.119189 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aafac6-301e-44ba-823a-63202a2077b0 None admin] RBAC: Authorizing `identity:get_project(project_id=4edef8e255584377bf94a9ac9712b4b7)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:54.120686 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aafac6-301e-44ba-823a-63202a2077b0 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:54.122705 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 75/153] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:08:53 2022] GET /identity/v3/projects/4edef8e255584377bf94a9ac9712b4b7 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:08:54.126013 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1746e0-6db3-4423-b094-4063e08f9253 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:54.126013 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1746e0-6db3-4423-b094-4063e08f9253 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:54.126013 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1746e0-6db3-4423-b094-4063e08f9253 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:54.126013 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 76/154] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:08:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:08:54.131706 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d26f76-49f8-4290-8280-3f778dc196e9 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:54.132232 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d26f76-49f8-4290-8280-3f778dc196e9 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:54.132605 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d26f76-49f8-4290-8280-3f778dc196e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:54.156632 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-79d26f76-49f8-4290-8280-3f778dc196e9 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:08:54.158180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-79d26f76-49f8-4290-8280-3f778dc196e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:08:54.181237 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6baba44-ac55-4717-ad45-689a95080fff None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:54.246477 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f23313-298e-4790-87d4-a7b385414b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:54.247559 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f23313-298e-4790-87d4-a7b385414b09 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:54.247838 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f23313-298e-4790-87d4-a7b385414b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:54.248107 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f23313-298e-4790-87d4-a7b385414b09 None admin] PATH_INFO: `/v3/projects/4edef8e255584377bf94a9ac9712b4b7` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:54.251172 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f23313-298e-4790-87d4-a7b385414b09 None admin] RBAC: Authorizing `identity:get_project(project_id=4edef8e255584377bf94a9ac9712b4b7)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:54.252686 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f23313-298e-4790-87d4-a7b385414b09 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:54.255129 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 79/155] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:08:54 2022] GET /identity/v3/projects/4edef8e255584377bf94a9ac9712b4b7 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:54.255489 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f23313-298e-4790-87d4-a7b385414b09 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:54.259313 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 77/156] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:08:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:08:54.279690 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d26f76-49f8-4290-8280-3f778dc196e9 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:54.309366 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:54.352813 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:54.403389 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f23313-298e-4790-87d4-a7b385414b09 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:54.441237 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d26f76-49f8-4290-8280-3f778dc196e9 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:54.461667 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0cfe5-ea45-40f1-9c5b-ca5de740dd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:54.462336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0cfe5-ea45-40f1-9c5b-ca5de740dd65 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:54.462614 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0cfe5-ea45-40f1-9c5b-ca5de740dd65 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:54.462868 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0cfe5-ea45-40f1-9c5b-ca5de740dd65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:54.463726 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0cfe5-ea45-40f1-9c5b-ca5de740dd65 None admin] RBAC: Adding query filter params (role.id=9e63b73b3ce24bb3ad214ecdb6132696, scope.project.id=4edef8e255584377bf94a9ac9712b4b7, user.id=ba8ddcedde1a4bee9ba699929814f673) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:08:54.464010 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0cfe5-ea45-40f1-9c5b-ca5de740dd65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:54.465515 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0cfe5-ea45-40f1-9c5b-ca5de740dd65 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:54.477703 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 80/157] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:08:54 2022] GET /identity/v3/role_assignments?user.id=ba8ddcedde1a4bee9ba699929814f673&scope.project.id=4edef8e255584377bf94a9ac9712b4b7&role.id=9e63b73b3ce24bb3ad214ecdb6132696 => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:54.477703 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0cfe5-ea45-40f1-9c5b-ca5de740dd65 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:54.507263 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc7918-42c3-494f-ba13-d99038786969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:54.508769 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc7918-42c3-494f-ba13-d99038786969 None admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:54.509262 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc7918-42c3-494f-ba13-d99038786969 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:54.509745 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc7918-42c3-494f-ba13-d99038786969 None admin] PATH_INFO: `/v3/projects/4edef8e255584377bf94a9ac9712b4b7/users/3fbe4915d35149f6ab1a29cdfb821f19/roles/6479235705f74061aff23df5c9f89e33` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:54.514292 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cc7918-42c3-494f-ba13-d99038786969 None admin] RBAC: Authorizing `identity:create_grant(project_id=4edef8e255584377bf94a9ac9712b4b7, user_id=3fbe4915d35149f6ab1a29cdfb821f19, role_id=6479235705f74061aff23df5c9f89e33)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:54.516010 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cc7918-42c3-494f-ba13-d99038786969 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:54.567657 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 78/158] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 03:08:54 2022] PUT /identity/v3/projects/4edef8e255584377bf94a9ac9712b4b7/users/3fbe4915d35149f6ab1a29cdfb821f19/roles/6479235705f74061aff23df5c9f89e33 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:08:54.637412 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:54.647313 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0cfe5-ea45-40f1-9c5b-ca5de740dd65 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:54.712773 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cbe25a-1d0f-432a-b51a-35f33a300085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:54.713656 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cbe25a-1d0f-432a-b51a-35f33a300085 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:54.713934 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cbe25a-1d0f-432a-b51a-35f33a300085 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:54.714207 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cbe25a-1d0f-432a-b51a-35f33a300085 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:54.739431 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cbe25a-1d0f-432a-b51a-35f33a300085 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:08:54.747603 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-74cbe25a-1d0f-432a-b51a-35f33a300085 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:08:54.747603 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:08:54.747603 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:08:54.747603 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:08:54.747603 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:08:54.747603 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:08:54.747603 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:08:54.747603 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:08:54.747603 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:08:54.747603 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:08:54.747603 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:08:54.747603 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:08:54.747603 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:08:54.747603 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:08:54.747603 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:08:54.747603 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:08:54.748538 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:08:54.748538 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:08:54.748538 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:08:54.748538 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:08:54.748538 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:08:54.748538 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:08:54.748538 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:08:54.748538 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:08:54.748538 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:08:54.748538 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:08:54.748538 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:08:54.748538 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:08:54.748538 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:08:54.748538 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:08:54.748538 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:08:54.748538 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:08:54.749397 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:08:54.749397 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:08:54.749397 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:08:54.749397 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:08:54.749397 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:08:54.749397 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:08:54.749397 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:08:54.749397 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:08:54.749397 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:08:54.749397 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:08:54.749397 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:08:54.749397 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:08:54.753391 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 81/159] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:08:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:08:54.754977 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc7918-42c3-494f-ba13-d99038786969 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:54.881587 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc7918-42c3-494f-ba13-d99038786969 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:54.884115 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:54.922569 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9039bd82-d743-4b69-9b1d-40f5f152d248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:54.923272 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039bd82-d743-4b69-9b1d-40f5f152d248 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:54.923551 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039bd82-d743-4b69-9b1d-40f5f152d248 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:54.923826 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039bd82-d743-4b69-9b1d-40f5f152d248 None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:54.924599 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9039bd82-d743-4b69-9b1d-40f5f152d248 None admin] RBAC: Adding query filter params (name=member) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:08:54.924905 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9039bd82-d743-4b69-9b1d-40f5f152d248 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:54.926378 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9039bd82-d743-4b69-9b1d-40f5f152d248 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:54.941511 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 79/160] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 03:08:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:54.949091 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cbe25a-1d0f-432a-b51a-35f33a300085 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:55.066281 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cbe25a-1d0f-432a-b51a-35f33a300085 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:55.111367 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aa651d-eb39-4e44-94ac-1876f66dd35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:55.112262 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa651d-eb39-4e44-94ac-1876f66dd35c None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:55.112548 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa651d-eb39-4e44-94ac-1876f66dd35c None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:55.112813 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aa651d-eb39-4e44-94ac-1876f66dd35c None admin] PATH_INFO: `/v3/users/ba8ddcedde1a4bee9ba699929814f673` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:55.117691 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aa651d-eb39-4e44-94ac-1876f66dd35c None admin] RBAC: Authorizing `identity:get_user(user_id=ba8ddcedde1a4bee9ba699929814f673)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:55.119276 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aa651d-eb39-4e44-94ac-1876f66dd35c None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:55.120996 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 82/161] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:08:54 2022] GET /identity/v3/users/ba8ddcedde1a4bee9ba699929814f673 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:55.126759 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9039bd82-d743-4b69-9b1d-40f5f152d248 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:55.162337 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0153d8c1-94b8-463f-81c4-8063ce6091a2 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:55.162519 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0153d8c1-94b8-463f-81c4-8063ce6091a2 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:55.162800 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0153d8c1-94b8-463f-81c4-8063ce6091a2 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:55.164698 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 83/162] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:08:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:08:55.169764 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc82829-9ac4-40ea-89ed-9fd0efb8ddf2 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:55.170048 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc82829-9ac4-40ea-89ed-9fd0efb8ddf2 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:55.170326 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc82829-9ac4-40ea-89ed-9fd0efb8ddf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:55.204015 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-adc82829-9ac4-40ea-89ed-9fd0efb8ddf2 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:08:55.205279 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-adc82829-9ac4-40ea-89ed-9fd0efb8ddf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:08:55.245372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:55.345637 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 84/163] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:08:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:08:55.346537 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9039bd82-d743-4b69-9b1d-40f5f152d248 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:55.347545 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e7e56-866d-4a95-a6f6-3f5d2fcf4e93 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:55.348017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e7e56-866d-4a95-a6f6-3f5d2fcf4e93 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:55.348474 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e7e56-866d-4a95-a6f6-3f5d2fcf4e93 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:55.350100 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 85/164] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:08:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:08:55.355673 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e605f5-3144-4ba4-ac87-18dd2a53a3c8 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:55.356230 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e605f5-3144-4ba4-ac87-18dd2a53a3c8 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:55.356692 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e605f5-3144-4ba4-ac87-18dd2a53a3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:55.378577 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-18e605f5-3144-4ba4-ac87-18dd2a53a3c8 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:08:55.380131 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-18e605f5-3144-4ba4-ac87-18dd2a53a3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:08:55.397727 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e0c087-9910-4e4e-9a7f-24597dfd5214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:55.398405 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0c087-9910-4e4e-9a7f-24597dfd5214 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:55.398689 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0c087-9910-4e4e-9a7f-24597dfd5214 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:55.398967 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0c087-9910-4e4e-9a7f-24597dfd5214 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:55.452538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:55.486471 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 86/165] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:08:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:08:55.487688 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e605f5-3144-4ba4-ac87-18dd2a53a3c8 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:55.500929 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e0c087-9910-4e4e-9a7f-24597dfd5214 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:55.502462 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e0c087-9910-4e4e-9a7f-24597dfd5214 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:55.609070 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 80/166] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:08:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:08:55.614004 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e0c087-9910-4e4e-9a7f-24597dfd5214 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:55.673916 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:55.722901 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e605f5-3144-4ba4-ac87-18dd2a53a3c8 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:55.741378 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:55.798852 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb7b77-1a42-42e7-8402-e06929e78114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:55.799658 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb7b77-1a42-42e7-8402-e06929e78114 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:55.799938 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb7b77-1a42-42e7-8402-e06929e78114 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:55.800209 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb7b77-1a42-42e7-8402-e06929e78114 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:55.808726 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e0c087-9910-4e4e-9a7f-24597dfd5214 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:55.833945 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb7b77-1a42-42e7-8402-e06929e78114 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:08:55.841906 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-32fb7b77-1a42-42e7-8402-e06929e78114 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:08:55.841906 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:08:55.841906 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:08:55.841906 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:08:55.841906 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:08:55.841906 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:08:55.841906 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:08:55.841906 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:08:55.841906 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:08:55.841906 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:08:55.841906 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:08:55.841906 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:08:55.841906 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:08:55.841906 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:08:55.841906 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:08:55.841906 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:08:55.842875 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:08:55.842875 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:08:55.842875 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:08:55.842875 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:08:55.842875 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:08:55.842875 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:08:55.842875 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:08:55.842875 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:08:55.842875 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:08:55.842875 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:08:55.842875 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:08:55.842875 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:08:55.842875 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:08:55.842875 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:08:55.842875 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:08:55.842875 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:08:55.843809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:08:55.843809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:08:55.843809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:08:55.843809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:08:55.843809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:08:55.843809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:08:55.843809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:08:55.843809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:08:55.843809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:08:55.843809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:08:55.843809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:08:55.843809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:08:55.849989 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 87/167] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 03:08:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:08:55.851182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb7b77-1a42-42e7-8402-e06929e78114 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:55.881089 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25e637-4922-4737-ac27-b9000eaef7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:55.881983 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25e637-4922-4737-ac27-b9000eaef7b7 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:55.882265 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25e637-4922-4737-ac27-b9000eaef7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:55.882536 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25e637-4922-4737-ac27-b9000eaef7b7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:55.908674 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac25e637-4922-4737-ac27-b9000eaef7b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:08:55.926383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-ac25e637-4922-4737-ac27-b9000eaef7b7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:08:55.926383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:08:55.926383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:08:55.926383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:08:55.926383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:08:55.926383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:08:55.926383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:08:55.926383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:08:55.926383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:08:55.926383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:08:55.926383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:08:55.926383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:08:55.926383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:08:55.926383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:08:55.926383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:08:55.926383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:08:55.927369 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:08:55.927369 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:08:55.927369 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:08:55.927369 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:08:55.927369 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:08:55.927369 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:08:55.927369 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:08:55.927369 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:08:55.927369 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:08:55.927369 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:08:55.927369 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:08:55.927369 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:08:55.927369 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:08:55.927369 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:08:55.927369 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:08:55.927369 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:08:55.928618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:08:55.928618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:08:55.928618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:08:55.928618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:08:55.928618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:08:55.928618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:08:55.928618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:08:55.928618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:08:55.928618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:08:55.928618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:08:55.928618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:08:55.928618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:08:55.928618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 81/168] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:08:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:08:55.928618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25e637-4922-4737-ac27-b9000eaef7b7 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:56.028204 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb7b77-1a42-42e7-8402-e06929e78114 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:56.105042 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d2ec6a-8cc7-425f-85fb-9dad2020f4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:56.105929 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2ec6a-8cc7-425f-85fb-9dad2020f4b2 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:56.106206 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2ec6a-8cc7-425f-85fb-9dad2020f4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:56.106478 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2ec6a-8cc7-425f-85fb-9dad2020f4b2 None admin] PATH_INFO: `/v3/projects/4edef8e255584377bf94a9ac9712b4b7` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:56.108639 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d2ec6a-8cc7-425f-85fb-9dad2020f4b2 None admin] RBAC: Authorizing `identity:get_project(project_id=4edef8e255584377bf94a9ac9712b4b7)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:56.110155 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d2ec6a-8cc7-425f-85fb-9dad2020f4b2 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:56.111883 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 88/169] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:08:55 2022] GET /identity/v3/projects/4edef8e255584377bf94a9ac9712b4b7 => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:56.112641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d2ec6a-8cc7-425f-85fb-9dad2020f4b2 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:56.132749 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25e637-4922-4737-ac27-b9000eaef7b7 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:56.149314 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:56.189313 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:56.202844 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3782d8-4224-42df-a161-52ba0e188776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:56.203520 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3782d8-4224-42df-a161-52ba0e188776 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:56.203801 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3782d8-4224-42df-a161-52ba0e188776 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:56.204073 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3782d8-4224-42df-a161-52ba0e188776 None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:56.204829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3782d8-4224-42df-a161-52ba0e188776 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:08:56.205158 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3782d8-4224-42df-a161-52ba0e188776 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:56.206558 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3782d8-4224-42df-a161-52ba0e188776 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:56.224564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 82/170] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:08:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:56.225463 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3782d8-4224-42df-a161-52ba0e188776 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:56.271719 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d2ec6a-8cc7-425f-85fb-9dad2020f4b2 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:56.344092 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e319ce53-aff8-443a-a6aa-9b39f3541854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:56.344762 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319ce53-aff8-443a-a6aa-9b39f3541854 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:56.345069 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319ce53-aff8-443a-a6aa-9b39f3541854 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:56.345343 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319ce53-aff8-443a-a6aa-9b39f3541854 None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:56.346101 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e319ce53-aff8-443a-a6aa-9b39f3541854 None admin] RBAC: Adding query filter params (name=member) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:08:56.346404 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e319ce53-aff8-443a-a6aa-9b39f3541854 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:56.347831 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e319ce53-aff8-443a-a6aa-9b39f3541854 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:56.358305 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 89/171] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 03:08:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:56.359203 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e319ce53-aff8-443a-a6aa-9b39f3541854 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:56.440025 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3782d8-4224-42df-a161-52ba0e188776 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:56.486130 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:56.505984 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6326953b-7ed3-4e48-a587-b4ac16201c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:56.506694 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326953b-7ed3-4e48-a587-b4ac16201c4d None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:56.507146 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326953b-7ed3-4e48-a587-b4ac16201c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:56.507456 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6326953b-7ed3-4e48-a587-b4ac16201c4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:56.508712 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6326953b-7ed3-4e48-a587-b4ac16201c4d None admin] RBAC: Adding query filter params (role.id=0129dd45ff5449f9957e8efdd01a55ed, scope.project.id=4edef8e255584377bf94a9ac9712b4b7, user.id=ba8ddcedde1a4bee9ba699929814f673) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:08:56.509075 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6326953b-7ed3-4e48-a587-b4ac16201c4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:56.510727 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6326953b-7ed3-4e48-a587-b4ac16201c4d None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:56.523583 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 83/172] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:08:56 2022] GET /identity/v3/role_assignments?user.id=ba8ddcedde1a4bee9ba699929814f673&scope.project.id=4edef8e255584377bf94a9ac9712b4b7&role.id=0129dd45ff5449f9957e8efdd01a55ed => generated 608 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:56.524808 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6326953b-7ed3-4e48-a587-b4ac16201c4d None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:56.564033 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e319ce53-aff8-443a-a6aa-9b39f3541854 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:56.627616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:56.677030 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e6c20-745e-4207-875c-2d6ba157971f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:56.678335 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e6c20-745e-4207-875c-2d6ba157971f None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:56.678906 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e6c20-745e-4207-875c-2d6ba157971f None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:56.679384 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e6c20-745e-4207-875c-2d6ba157971f None admin] PATH_INFO: `/v3/users/4a96d7acf93e46c7ac5700560db36dbe` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:56.686120 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e6c20-745e-4207-875c-2d6ba157971f None admin] RBAC: Authorizing `identity:get_user(user_id=4a96d7acf93e46c7ac5700560db36dbe)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:56.688023 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e6c20-745e-4207-875c-2d6ba157971f None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:56.690508 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 90/173] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:08:56 2022] GET /identity/v3/users/4a96d7acf93e46c7ac5700560db36dbe => generated 247 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:56.696037 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e6c20-745e-4207-875c-2d6ba157971f None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:56.751565 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6326953b-7ed3-4e48-a587-b4ac16201c4d None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:56.822116 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528ec86-26fd-44d7-aa10-5eb902b1e8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:56.822937 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528ec86-26fd-44d7-aa10-5eb902b1e8a5 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:56.823218 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528ec86-26fd-44d7-aa10-5eb902b1e8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:56.823495 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528ec86-26fd-44d7-aa10-5eb902b1e8a5 None admin] PATH_INFO: `/v3/users/ba8ddcedde1a4bee9ba699929814f673` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:56.827770 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e528ec86-26fd-44d7-aa10-5eb902b1e8a5 None admin] RBAC: Authorizing `identity:get_user(user_id=ba8ddcedde1a4bee9ba699929814f673)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:56.827770 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e528ec86-26fd-44d7-aa10-5eb902b1e8a5 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:56.827948 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 84/174] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:08:56 2022] GET /identity/v3/users/ba8ddcedde1a4bee9ba699929814f673 => generated 275 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:56.833863 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528ec86-26fd-44d7-aa10-5eb902b1e8a5 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:56.927227 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e6c20-745e-4207-875c-2d6ba157971f None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:56.999190 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a4f95-6891-4049-aae1-1dd9d5d0e4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:56.999863 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a4f95-6891-4049-aae1-1dd9d5d0e4dc None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:57.000149 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a4f95-6891-4049-aae1-1dd9d5d0e4dc None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:57.000428 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a4f95-6891-4049-aae1-1dd9d5d0e4dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:57.021776 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:57.033725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528ec86-26fd-44d7-aa10-5eb902b1e8a5 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:57.065570 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:57.087685 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c46e147-c897-4ae1-a83d-ce9a0e7a3cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:57.088743 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46e147-c897-4ae1-a83d-ce9a0e7a3cfd None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:57.089260 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46e147-c897-4ae1-a83d-ce9a0e7a3cfd None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:57.089728 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46e147-c897-4ae1-a83d-ce9a0e7a3cfd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:57.152424 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a4f95-6891-4049-aae1-1dd9d5d0e4dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:57.153969 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a4f95-6891-4049-aae1-1dd9d5d0e4dc None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:57.183661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c46e147-c897-4ae1-a83d-ce9a0e7a3cfd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:57.190015 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c46e147-c897-4ae1-a83d-ce9a0e7a3cfd None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:57.327783 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 85/175] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:08:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:08:57.329288 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab666097-84a6-489e-a432-503fd0cd8dee None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:57.329576 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab666097-84a6-489e-a432-503fd0cd8dee None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:57.329917 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab666097-84a6-489e-a432-503fd0cd8dee None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:57.331460 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 86/176] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:08:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:08:57.338101 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab666097-84a6-489e-a432-503fd0cd8dee None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:57.373381 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:57.391199 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 91/177] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:08:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 696 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 03:08:57.405924 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e3f59-057c-453e-b20f-7d9bf90cb9b3 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:57.406243 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e3f59-057c-453e-b20f-7d9bf90cb9b3 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:57.406494 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e3f59-057c-453e-b20f-7d9bf90cb9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:57.451299 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-a15e3f59-057c-453e-b20f-7d9bf90cb9b3 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:08:57.452515 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-a15e3f59-057c-453e-b20f-7d9bf90cb9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:08:57.521658 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:57.531041 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab666097-84a6-489e-a432-503fd0cd8dee None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:57.570304 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 92/178] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:08:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:08:57.571144 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15e3f59-057c-453e-b20f-7d9bf90cb9b3 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:57.604902 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a37985-ef19-4d0d-bfa5-c9ba7dbc1068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:57.606168 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a37985-ef19-4d0d-bfa5-c9ba7dbc1068 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:57.606650 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a37985-ef19-4d0d-bfa5-c9ba7dbc1068 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:57.607132 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a37985-ef19-4d0d-bfa5-c9ba7dbc1068 None admin] PATH_INFO: `/v3/projects/0f4c723f76b04fab9f9ac3617cf3fb8b` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:57.609796 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a37985-ef19-4d0d-bfa5-c9ba7dbc1068 None admin] RBAC: Authorizing `identity:get_project(project_id=0f4c723f76b04fab9f9ac3617cf3fb8b)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:57.611377 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a37985-ef19-4d0d-bfa5-c9ba7dbc1068 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:57.613297 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 87/179] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:08:57 2022] GET /identity/v3/projects/0f4c723f76b04fab9f9ac3617cf3fb8b => generated 312 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:57.620446 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a37985-ef19-4d0d-bfa5-c9ba7dbc1068 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:57.699080 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15e3f59-057c-453e-b20f-7d9bf90cb9b3 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:57.744504 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:57.748541 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c0aad-3ea2-4802-9d38-737a20ded520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:57.749600 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c0aad-3ea2-4802-9d38-737a20ded520 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:57.750120 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c0aad-3ea2-4802-9d38-737a20ded520 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:57.750562 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c0aad-3ea2-4802-9d38-737a20ded520 None admin] PATH_INFO: `/v3/projects/4edef8e255584377bf94a9ac9712b4b7` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:57.752454 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038c0aad-3ea2-4802-9d38-737a20ded520 None admin] RBAC: Authorizing `identity:get_project(project_id=4edef8e255584377bf94a9ac9712b4b7)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:57.754512 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038c0aad-3ea2-4802-9d38-737a20ded520 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:57.757151 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 93/180] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:08:57 2022] GET /identity/v3/projects/4edef8e255584377bf94a9ac9712b4b7 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:57.758070 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c0aad-3ea2-4802-9d38-737a20ded520 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:57.758930 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a37985-ef19-4d0d-bfa5-c9ba7dbc1068 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:57.797058 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f964e8-15ea-4fe9-a419-9f4d185b4797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:57.798033 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f964e8-15ea-4fe9-a419-9f4d185b4797 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:57.798383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f964e8-15ea-4fe9-a419-9f4d185b4797 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:57.798721 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f964e8-15ea-4fe9-a419-9f4d185b4797 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:57.811418 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f964e8-15ea-4fe9-a419-9f4d185b4797 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:08:57.819937 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-71f964e8-15ea-4fe9-a419-9f4d185b4797 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:08:57.819937 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:08:57.819937 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:08:57.819937 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:08:57.819937 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:08:57.819937 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:08:57.819937 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:08:57.819937 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:08:57.819937 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:08:57.819937 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:08:57.819937 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:08:57.819937 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:08:57.819937 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:08:57.819937 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:08:57.819937 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:08:57.819937 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:08:57.821072 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:08:57.821072 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:08:57.821072 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:08:57.821072 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:08:57.821072 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:08:57.821072 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:08:57.821072 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:08:57.821072 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:08:57.821072 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:08:57.821072 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:08:57.821072 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:08:57.821072 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:08:57.821072 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:08:57.821072 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:08:57.821072 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:08:57.821072 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:08:57.822042 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:08:57.822042 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:08:57.822042 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:08:57.822042 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:08:57.822042 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:08:57.822042 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:08:57.822042 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:08:57.822042 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:08:57.822042 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:08:57.822042 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:08:57.822042 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:08:57.822042 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:08:57.824026 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 88/181] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 03:08:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:08:57.825307 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df73cea0-cf05-4083-a042-be7ecb804b59 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:57.825688 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df73cea0-cf05-4083-a042-be7ecb804b59 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:57.826069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df73cea0-cf05-4083-a042-be7ecb804b59 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:57.827672 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 89/182] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:08:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:08:57.828365 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df73cea0-cf05-4083-a042-be7ecb804b59 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:57.837375 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:57.890445 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c0aad-3ea2-4802-9d38-737a20ded520 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:57.930956 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d096a84-d75d-4ed4-b0a1-80040c1eae59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:57.931616 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d096a84-d75d-4ed4-b0a1-80040c1eae59 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:57.931895 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d096a84-d75d-4ed4-b0a1-80040c1eae59 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:57.932166 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d096a84-d75d-4ed4-b0a1-80040c1eae59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:57.933017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d096a84-d75d-4ed4-b0a1-80040c1eae59 None admin] RBAC: Adding query filter params (role.id=0129dd45ff5449f9957e8efdd01a55ed, scope.project.id=0f4c723f76b04fab9f9ac3617cf3fb8b, user.id=ba8ddcedde1a4bee9ba699929814f673) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:08:57.933328 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d096a84-d75d-4ed4-b0a1-80040c1eae59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:57.934782 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d096a84-d75d-4ed4-b0a1-80040c1eae59 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:57.948992 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 94/183] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:08:57 2022] GET /identity/v3/role_assignments?user.id=ba8ddcedde1a4bee9ba699929814f673&scope.project.id=0f4c723f76b04fab9f9ac3617cf3fb8b&role.id=0129dd45ff5449f9957e8efdd01a55ed => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:57.948992 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d096a84-d75d-4ed4-b0a1-80040c1eae59 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:57.950600 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df73cea0-cf05-4083-a042-be7ecb804b59 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:57.995044 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90199dc2-bcfd-41af-9afa-7d4309885442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:57.995878 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90199dc2-bcfd-41af-9afa-7d4309885442 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:57.996253 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90199dc2-bcfd-41af-9afa-7d4309885442 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:57.996593 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90199dc2-bcfd-41af-9afa-7d4309885442 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:57.997573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90199dc2-bcfd-41af-9afa-7d4309885442 None admin] RBAC: Adding query filter params (role.id=b4c524dd7bb44e94bd3b42d86e1cdf42, scope.project.id=4edef8e255584377bf94a9ac9712b4b7, user.id=4a96d7acf93e46c7ac5700560db36dbe) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:08:57.997954 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90199dc2-bcfd-41af-9afa-7d4309885442 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:57.999724 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90199dc2-bcfd-41af-9afa-7d4309885442 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:58.006829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 90/184] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:08:57 2022] GET /identity/v3/role_assignments?user.id=4a96d7acf93e46c7ac5700560db36dbe&scope.project.id=4edef8e255584377bf94a9ac9712b4b7&role.id=b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:58.015495 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4cad3-4222-4391-a381-31a7e2f5ad5d None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:58.015866 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4cad3-4222-4391-a381-31a7e2f5ad5d None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:58.016045 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4cad3-4222-4391-a381-31a7e2f5ad5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:58.035941 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-37f4cad3-4222-4391-a381-31a7e2f5ad5d None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:08:58.037264 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-37f4cad3-4222-4391-a381-31a7e2f5ad5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:08:58.046796 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:58.087209 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d096a84-d75d-4ed4-b0a1-80040c1eae59 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:58.089489 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:58.120140 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 91/185] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:08:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:08:58.129323 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829bd85-7418-4f78-9047-a343cfd9682b None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:58.129649 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829bd85-7418-4f78-9047-a343cfd9682b None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:58.129924 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829bd85-7418-4f78-9047-a343cfd9682b None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:58.131753 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 92/186] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:08:58 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:08:58.133464 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c74be9-694f-49f5-b0bc-0434ba11e7c8 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:58.133743 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c74be9-694f-49f5-b0bc-0434ba11e7c8 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:58.134069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c74be9-694f-49f5-b0bc-0434ba11e7c8 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:58.134598 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c5f83-0c32-4405-8bd3-cf754ff0a168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:58.135524 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c5f83-0c32-4405-8bd3-cf754ff0a168 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:58.139837 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 93/187] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:08:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:08:58.140694 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c5f83-0c32-4405-8bd3-cf754ff0a168 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:58.141222 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c5f83-0c32-4405-8bd3-cf754ff0a168 None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:58.142255 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c5f83-0c32-4405-8bd3-cf754ff0a168 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:08:58.142674 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1071301-c9f2-4520-938d-939c4b923b5a None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:58.142950 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1071301-c9f2-4520-938d-939c4b923b5a None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:58.143264 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1071301-c9f2-4520-938d-939c4b923b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:58.143622 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c5f83-0c32-4405-8bd3-cf754ff0a168 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:58.145769 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c5f83-0c32-4405-8bd3-cf754ff0a168 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:58.157063 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 95/188] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:08:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:58.158747 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6767b311-5773-4f8a-9ce3-a1686c22527c None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:58.159241 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6767b311-5773-4f8a-9ce3-a1686c22527c None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:58.159708 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6767b311-5773-4f8a-9ce3-a1686c22527c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:58.175030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-c1071301-c9f2-4520-938d-939c4b923b5a None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:08:58.176258 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-c1071301-c9f2-4520-938d-939c4b923b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:08:58.180574 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-6767b311-5773-4f8a-9ce3-a1686c22527c None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:08:58.182157 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-6767b311-5773-4f8a-9ce3-a1686c22527c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:08:58.260979 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 94/189] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:08:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:08:58.262873 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1071301-c9f2-4520-938d-939c4b923b5a None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:58.283629 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 96/190] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:08:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:08:58.286416 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6767b311-5773-4f8a-9ce3-a1686c22527c None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:58.357250 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:58.387030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1071301-c9f2-4520-938d-939c4b923b5a None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:58.390278 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:58.397860 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6767b311-5773-4f8a-9ce3-a1686c22527c None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:58.421725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06467b08-5831-4b6b-abd3-00565acc1474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:58.422600 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06467b08-5831-4b6b-abd3-00565acc1474 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:58.422868 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06467b08-5831-4b6b-abd3-00565acc1474 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:58.423132 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06467b08-5831-4b6b-abd3-00565acc1474 None admin] PATH_INFO: `/v3/users/8b1afed4cb0342a783c95efb73a8b869` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:58.432841 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d5761-52aa-4a59-9206-3316e7861b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:58.433684 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d5761-52aa-4a59-9206-3316e7861b3f None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:58.433970 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d5761-52aa-4a59-9206-3316e7861b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:58.434278 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046d5761-52aa-4a59-9206-3316e7861b3f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:58.435395 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046d5761-52aa-4a59-9206-3316e7861b3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:58.436661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06467b08-5831-4b6b-abd3-00565acc1474 None admin] RBAC: Authorizing `identity:get_user(user_id=8b1afed4cb0342a783c95efb73a8b869)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:58.436855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046d5761-52aa-4a59-9206-3316e7861b3f None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:58.438170 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06467b08-5831-4b6b-abd3-00565acc1474 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:58.438534 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 97/191] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:08:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:58.439293 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d5761-52aa-4a59-9206-3316e7861b3f None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:58.439846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 95/192] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:08:58 2022] GET /identity/v3/users/8b1afed4cb0342a783c95efb73a8b869 => generated 283 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:58.441694 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06467b08-5831-4b6b-abd3-00565acc1474 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:58.544799 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06467b08-5831-4b6b-abd3-00565acc1474 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:58.546997 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046d5761-52aa-4a59-9206-3316e7861b3f None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:58.573086 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:58.579037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c7189b-f216-4438-b7c6-fd0947480f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:58.579844 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7189b-f216-4438-b7c6-fd0947480f46 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:58.580121 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7189b-f216-4438-b7c6-fd0947480f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:58.580393 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7189b-f216-4438-b7c6-fd0947480f46 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:58.581730 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edef461-4bf9-4808-809f-9c6a1b346009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:58.582535 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edef461-4bf9-4808-809f-9c6a1b346009 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:58.582805 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edef461-4bf9-4808-809f-9c6a1b346009 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:58.583071 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edef461-4bf9-4808-809f-9c6a1b346009 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:58.591249 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c7189b-f216-4438-b7c6-fd0947480f46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:08:58.593895 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edef461-4bf9-4808-809f-9c6a1b346009 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:08:58.599048 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-36c7189b-f216-4438-b7c6-fd0947480f46 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:08:58.599048 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:08:58.599048 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:08:58.599048 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:08:58.599048 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:08:58.599048 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:08:58.599048 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:08:58.599048 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:08:58.599048 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:08:58.599048 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:08:58.599048 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:08:58.599048 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:08:58.599048 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:08:58.599048 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:08:58.599048 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:08:58.599048 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:08:58.599878 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:08:58.599878 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:08:58.599878 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:08:58.599878 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:08:58.599878 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:08:58.599878 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:08:58.599878 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:08:58.599878 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:08:58.599878 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:08:58.599878 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:08:58.599878 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:08:58.599878 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:08:58.599878 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:08:58.599878 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:08:58.599878 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:08:58.599878 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:08:58.601531 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:08:58.601531 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:08:58.601531 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:08:58.601531 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:08:58.601531 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:08:58.601531 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:08:58.601531 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:08:58.601531 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:08:58.601531 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:08:58.601531 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:08:58.601531 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:08:58.601531 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:08:58.601531 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 96/193] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:08:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:08:58.601531 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c7189b-f216-4438-b7c6-fd0947480f46 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:58.602225 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-1edef461-4bf9-4808-809f-9c6a1b346009 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:08:58.602225 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:08:58.602225 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:08:58.602225 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:08:58.602225 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:08:58.602225 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:08:58.602225 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:08:58.602225 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:08:58.602225 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:08:58.602225 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:08:58.602225 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:08:58.602225 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:08:58.602225 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:08:58.602225 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:08:58.602225 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:08:58.602225 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:08:58.603006 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:08:58.603006 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:08:58.603006 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:08:58.603006 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:08:58.603006 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:08:58.603006 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:08:58.603006 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:08:58.603006 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:08:58.603006 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:08:58.603006 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:08:58.603006 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:08:58.603006 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:08:58.603006 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:08:58.603006 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:08:58.603006 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:08:58.603006 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:08:58.603774 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:08:58.603774 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:08:58.603774 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:08:58.603774 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:08:58.603774 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:08:58.603774 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:08:58.603774 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:08:58.603774 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:08:58.603774 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:08:58.603774 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:08:58.603774 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:08:58.603774 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:08:58.603774 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 98/194] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 03:08:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:08:58.604394 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edef461-4bf9-4808-809f-9c6a1b346009 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:58.631135 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:58.706057 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c7189b-f216-4438-b7c6-fd0947480f46 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:58.708185 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edef461-4bf9-4808-809f-9c6a1b346009 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:58.740358 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef4c24-8afa-4fca-a652-895aca7b16a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:58.741252 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef4c24-8afa-4fca-a652-895aca7b16a3 None admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:58.741365 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe7f838-2201-4ace-933c-d4f764a02e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:58.741898 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef4c24-8afa-4fca-a652-895aca7b16a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:58.741988 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7f838-2201-4ace-933c-d4f764a02e7a None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:58.742209 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef4c24-8afa-4fca-a652-895aca7b16a3 None admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:58.742340 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7f838-2201-4ace-933c-d4f764a02e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:58.742540 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7f838-2201-4ace-933c-d4f764a02e7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:58.742917 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef4c24-8afa-4fca-a652-895aca7b16a3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:58.744414 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef4c24-8afa-4fca-a652-895aca7b16a3 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:58.772094 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 97/195] 10.222.0.50 () {60 vars in 1132 bytes} [Fri May 27 03:08:58 2022] POST /identity/v3/users => generated 297 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:08:58.773921 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef4c24-8afa-4fca-a652-895aca7b16a3 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:58.816630 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe7f838-2201-4ace-933c-d4f764a02e7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:58.818629 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe7f838-2201-4ace-933c-d4f764a02e7a None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:58.833550 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:58.846210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:58.897965 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef4c24-8afa-4fca-a652-895aca7b16a3 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:58.935470 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 99/196] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:08:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:08:58.936347 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe7f838-2201-4ace-933c-d4f764a02e7a None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:58.941257 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48df62-464f-4d47-ad50-fc39c985539e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:58.942107 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48df62-464f-4d47-ad50-fc39c985539e None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:58.942392 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48df62-464f-4d47-ad50-fc39c985539e None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:58.942666 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a48df62-464f-4d47-ad50-fc39c985539e None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:58.943451 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a48df62-464f-4d47-ad50-fc39c985539e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:08:58.943760 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a48df62-464f-4d47-ad50-fc39c985539e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:58.945501 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a48df62-464f-4d47-ad50-fc39c985539e None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:58.956482 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 98/197] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 03:08:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:58.958642 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48df62-464f-4d47-ad50-fc39c985539e None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:59.056134 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe7f838-2201-4ace-933c-d4f764a02e7a None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:59.084344 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:59.094830 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a48df62-464f-4d47-ad50-fc39c985539e None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:59.099398 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4f67d7-434f-4846-ae3e-00d0fccebaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:59.100343 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f67d7-434f-4846-ae3e-00d0fccebaef None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:59.101014 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f67d7-434f-4846-ae3e-00d0fccebaef None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:59.101623 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4f67d7-434f-4846-ae3e-00d0fccebaef None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:59.102651 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4f67d7-434f-4846-ae3e-00d0fccebaef None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:08:59.103163 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4f67d7-434f-4846-ae3e-00d0fccebaef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:59.104700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:59.105647 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4f67d7-434f-4846-ae3e-00d0fccebaef None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:59.115410 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 100/198] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:08:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:59.116877 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4f67d7-434f-4846-ae3e-00d0fccebaef None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:59.133908 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c180684-3e31-41dd-b223-e7eb1feaa775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:59.134789 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c180684-3e31-41dd-b223-e7eb1feaa775 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:59.135071 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c180684-3e31-41dd-b223-e7eb1feaa775 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:59.135345 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c180684-3e31-41dd-b223-e7eb1feaa775 None admin] PATH_INFO: `/v3/projects/5e327e3411214877a9f427330ef40dc5` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:59.136535 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c180684-3e31-41dd-b223-e7eb1feaa775 None admin] RBAC: Authorizing `identity:get_project(project_id=5e327e3411214877a9f427330ef40dc5)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:59.138579 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c180684-3e31-41dd-b223-e7eb1feaa775 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:59.139778 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 99/199] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:08:58 2022] GET /identity/v3/projects/5e327e3411214877a9f427330ef40dc5 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:59.140614 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c180684-3e31-41dd-b223-e7eb1feaa775 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:59.248179 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4f67d7-434f-4846-ae3e-00d0fccebaef None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:59.265828 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c180684-3e31-41dd-b223-e7eb1feaa775 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:59.284071 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81627527-562e-4e6f-99b0-36f8864d6e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:59.285974 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81627527-562e-4e6f-99b0-36f8864d6e48 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:59.286270 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81627527-562e-4e6f-99b0-36f8864d6e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:59.286546 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81627527-562e-4e6f-99b0-36f8864d6e48 None admin] PATH_INFO: `/v3/users/3fbe4915d35149f6ab1a29cdfb821f19` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:59.288027 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81627527-562e-4e6f-99b0-36f8864d6e48 None admin] RBAC: Authorizing `identity:get_user(user_id=3fbe4915d35149f6ab1a29cdfb821f19)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:59.290183 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81627527-562e-4e6f-99b0-36f8864d6e48 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:59.292577 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 101/200] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:08:59 2022] GET /identity/v3/users/3fbe4915d35149f6ab1a29cdfb821f19 => generated 289 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:59.293294 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81627527-562e-4e6f-99b0-36f8864d6e48 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:59.317827 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4dab68-7044-4d94-a4c8-31312603ba42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:59.319517 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4dab68-7044-4d94-a4c8-31312603ba42 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:59.319971 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4dab68-7044-4d94-a4c8-31312603ba42 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:59.320384 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4dab68-7044-4d94-a4c8-31312603ba42 None admin] PATH_INFO: `/v3/users/8b1afed4cb0342a783c95efb73a8b869` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:59.322349 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4dab68-7044-4d94-a4c8-31312603ba42 None admin] RBAC: Authorizing `identity:get_user(user_id=8b1afed4cb0342a783c95efb73a8b869)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:59.324563 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4dab68-7044-4d94-a4c8-31312603ba42 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:59.327601 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 100/201] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:08:59 2022] GET /identity/v3/users/8b1afed4cb0342a783c95efb73a8b869 => generated 283 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:59.329314 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4dab68-7044-4d94-a4c8-31312603ba42 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:59.361556 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:59.393346 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:59.426657 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81627527-562e-4e6f-99b0-36f8864d6e48 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:59.464121 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4dab68-7044-4d94-a4c8-31312603ba42 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:59.468087 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b84ddd-9155-453d-a861-277884903c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:59.469031 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b84ddd-9155-453d-a861-277884903c16 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:59.469510 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b84ddd-9155-453d-a861-277884903c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:59.469988 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b84ddd-9155-453d-a861-277884903c16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:59.471017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b84ddd-9155-453d-a861-277884903c16 None admin] RBAC: Adding query filter params (role.id=b4c524dd7bb44e94bd3b42d86e1cdf42, scope.project.id=5e327e3411214877a9f427330ef40dc5, user.id=8b1afed4cb0342a783c95efb73a8b869) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:08:59.471542 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b84ddd-9155-453d-a861-277884903c16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:59.473242 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b84ddd-9155-453d-a861-277884903c16 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:59.479807 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 102/202] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:08:59 2022] GET /identity/v3/role_assignments?user.id=8b1afed4cb0342a783c95efb73a8b869&scope.project.id=5e327e3411214877a9f427330ef40dc5&role.id=b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:08:59.486187 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b84ddd-9155-453d-a861-277884903c16 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:08:59.501730 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3633fefc-3baa-445c-a509-2079e61ca68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:59.502307 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633fefc-3baa-445c-a509-2079e61ca68f None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:59.502585 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633fefc-3baa-445c-a509-2079e61ca68f None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:59.502864 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633fefc-3baa-445c-a509-2079e61ca68f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:59.593873 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3633fefc-3baa-445c-a509-2079e61ca68f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:59.595407 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3633fefc-3baa-445c-a509-2079e61ca68f None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:59.699727 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:59.707738 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b84ddd-9155-453d-a861-277884903c16 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:08:59.732455 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:08:59.759612 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dca155-4fa1-4ad3-a009-80ea4f8ee8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:08:59.760255 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dca155-4fa1-4ad3-a009-80ea4f8ee8f1 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:08:59.760540 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dca155-4fa1-4ad3-a009-80ea4f8ee8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:08:59.760864 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dca155-4fa1-4ad3-a009-80ea4f8ee8f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:08:59.839626 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dca155-4fa1-4ad3-a009-80ea4f8ee8f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:08:59.841174 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dca155-4fa1-4ad3-a009-80ea4f8ee8f1 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:08:59.846986 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 101/203] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:08:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:08:59.853165 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3633fefc-3baa-445c-a509-2079e61ca68f None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:00.017856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3633fefc-3baa-445c-a509-2079e61ca68f None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:00.057131 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6daa9d-4720-443e-b0bd-fabe7fb6c718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:00.057843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 103/204] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:08:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:09:00.058014 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6daa9d-4720-443e-b0bd-fabe7fb6c718 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:00.058290 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6daa9d-4720-443e-b0bd-fabe7fb6c718 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:00.058563 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6daa9d-4720-443e-b0bd-fabe7fb6c718 None admin] PATH_INFO: `/v3/projects/4edef8e255584377bf94a9ac9712b4b7` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:00.059839 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6daa9d-4720-443e-b0bd-fabe7fb6c718 None admin] RBAC: Authorizing `identity:get_project(project_id=4edef8e255584377bf94a9ac9712b4b7)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:00.061331 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6daa9d-4720-443e-b0bd-fabe7fb6c718 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:00.063325 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 102/205] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:08:59 2022] GET /identity/v3/projects/4edef8e255584377bf94a9ac9712b4b7 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:00.064090 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dca155-4fa1-4ad3-a009-80ea4f8ee8f1 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:00.068777 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6daa9d-4720-443e-b0bd-fabe7fb6c718 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:00.084280 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:00.103574 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:00.231609 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6daa9d-4720-443e-b0bd-fabe7fb6c718 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:00.238733 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dca155-4fa1-4ad3-a009-80ea4f8ee8f1 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:00.266248 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d4943d-1418-4f8e-ab4d-a4de87505d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:00.273041 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d4943d-1418-4f8e-ab4d-a4de87505d34 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:00.273640 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d4943d-1418-4f8e-ab4d-a4de87505d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:00.274107 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d4943d-1418-4f8e-ab4d-a4de87505d34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:00.275176 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d4943d-1418-4f8e-ab4d-a4de87505d34 None admin] RBAC: Adding query filter params (role.id=6479235705f74061aff23df5c9f89e33, scope.project.id=4edef8e255584377bf94a9ac9712b4b7, user.id=3fbe4915d35149f6ab1a29cdfb821f19) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:00.275687 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d4943d-1418-4f8e-ab4d-a4de87505d34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:00.277410 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d4943d-1418-4f8e-ab4d-a4de87505d34 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:00.285425 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 103/206] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:09:00 2022] GET /identity/v3/role_assignments?user.id=3fbe4915d35149f6ab1a29cdfb821f19&scope.project.id=4edef8e255584377bf94a9ac9712b4b7&role.id=6479235705f74061aff23df5c9f89e33 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:00.311352 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91bfec-1046-4abd-bfbb-7b3d055870d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:00.312255 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91bfec-1046-4abd-bfbb-7b3d055870d4 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:00.312527 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91bfec-1046-4abd-bfbb-7b3d055870d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:00.312794 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91bfec-1046-4abd-bfbb-7b3d055870d4 None admin] PATH_INFO: `/v3/projects/5e327e3411214877a9f427330ef40dc5` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:00.314083 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91bfec-1046-4abd-bfbb-7b3d055870d4 None admin] RBAC: Authorizing `identity:get_project(project_id=5e327e3411214877a9f427330ef40dc5)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:00.315613 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91bfec-1046-4abd-bfbb-7b3d055870d4 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:00.317805 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 104/207] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:09:00 2022] GET /identity/v3/projects/5e327e3411214877a9f427330ef40dc5 => generated 302 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:00.324070 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d4943d-1418-4f8e-ab4d-a4de87505d34 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:00.426571 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:00.476996 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d4943d-1418-4f8e-ab4d-a4de87505d34 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:00.511160 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0963b53-8411-4663-ba93-c41c16d35803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:00.512054 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0963b53-8411-4663-ba93-c41c16d35803 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:00.512337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0963b53-8411-4663-ba93-c41c16d35803 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:00.512613 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0963b53-8411-4663-ba93-c41c16d35803 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:00.513553 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0963b53-8411-4663-ba93-c41c16d35803 None admin] RBAC: Adding query filter params (role.id=9e63b73b3ce24bb3ad214ecdb6132696, scope.project.id=5e327e3411214877a9f427330ef40dc5, user.id=8b1afed4cb0342a783c95efb73a8b869) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:00.513867 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0963b53-8411-4663-ba93-c41c16d35803 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:00.516975 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0963b53-8411-4663-ba93-c41c16d35803 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:00.522190 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 104/208] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:09:00 2022] GET /identity/v3/role_assignments?user.id=8b1afed4cb0342a783c95efb73a8b869&scope.project.id=5e327e3411214877a9f427330ef40dc5&role.id=9e63b73b3ce24bb3ad214ecdb6132696 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:01.336174 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a10fb-d043-4d51-833e-1111394dfff3 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:01.336731 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a10fb-d043-4d51-833e-1111394dfff3 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:01.336731 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a10fb-d043-4d51-833e-1111394dfff3 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:01.338106 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 105/209] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:01.344213 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdb33a-2715-4ec0-9b1a-3def5dee821f None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:01.344213 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdb33a-2715-4ec0-9b1a-3def5dee821f None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:01.344213 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdb33a-2715-4ec0-9b1a-3def5dee821f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:01.361820 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-56cdb33a-2715-4ec0-9b1a-3def5dee821f None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:01.363280 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-56cdb33a-2715-4ec0-9b1a-3def5dee821f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:01.430992 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 105/210] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:09:01.439129 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c739f4-b7f9-4c85-a83c-5c61e5237823 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:01.439410 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c739f4-b7f9-4c85-a83c-5c61e5237823 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:01.439682 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c739f4-b7f9-4c85-a83c-5c61e5237823 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:01.441242 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 106/211] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:01.449530 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497b515-b28e-43ce-81ae-adb413c0f880 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:01.449816 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497b515-b28e-43ce-81ae-adb413c0f880 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:01.450090 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497b515-b28e-43ce-81ae-adb413c0f880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:01.461399 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c739f4-b7f9-4c85-a83c-5c61e5237823 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:01.468036 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-3497b515-b28e-43ce-81ae-adb413c0f880 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:01.469261 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-3497b515-b28e-43ce-81ae-adb413c0f880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:01.553067 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:01.554134 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 106/212] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:01.555782 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:01.580554 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3497b515-b28e-43ce-81ae-adb413c0f880 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:01.581991 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c739f4-b7f9-4c85-a83c-5c61e5237823 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:01.618021 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e9ea7c-08f9-4c73-b7fb-b7a33db6645e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:01.619151 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9ea7c-08f9-4c73-b7fb-b7a33db6645e None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:01.619600 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9ea7c-08f9-4c73-b7fb-b7a33db6645e None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:01.620035 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e9ea7c-08f9-4c73-b7fb-b7a33db6645e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:01.631050 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e9ea7c-08f9-4c73-b7fb-b7a33db6645e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:01.639453 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-e3e9ea7c-08f9-4c73-b7fb-b7a33db6645e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:09:01.639453 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:01.639453 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:01.639453 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:01.639453 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:01.639453 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:01.639453 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:01.639453 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:01.639453 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:01.639453 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:01.639453 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:01.639453 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:01.639453 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:01.639453 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:01.639453 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:01.639453 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:01.640461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:01.640461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:01.640461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:01.640461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:01.640461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:01.640461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:01.640461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:01.640461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:01.640461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:01.640461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:01.640461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:01.640461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:01.640461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:01.640461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:01.640461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:01.640461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:01.641274 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:01.641274 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:01.641274 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:01.641274 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:01.641274 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:01.641274 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:01.641274 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:01.641274 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:01.641274 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:01.641274 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:01.641274 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:09:01.641274 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:01.645670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 107/213] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:09:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:09:01.652592 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e9ea7c-08f9-4c73-b7fb-b7a33db6645e None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:01.700205 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3497b515-b28e-43ce-81ae-adb413c0f880 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:01.736295 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea1d04-d470-4170-8b79-d2a4227b4670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:01.737409 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea1d04-d470-4170-8b79-d2a4227b4670 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:01.737832 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea1d04-d470-4170-8b79-d2a4227b4670 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:01.738273 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea1d04-d470-4170-8b79-d2a4227b4670 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:01.739734 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ea1d04-d470-4170-8b79-d2a4227b4670 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:01.741781 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ea1d04-d470-4170-8b79-d2a4227b4670 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:01.748560 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 107/214] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:01.749724 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea1d04-d470-4170-8b79-d2a4227b4670 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:01.786809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e9ea7c-08f9-4c73-b7fb-b7a33db6645e None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:01.823012 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a3165c-81a2-48ff-a0a0-aefefa12c7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:01.824073 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3165c-81a2-48ff-a0a0-aefefa12c7c1 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:01.824541 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3165c-81a2-48ff-a0a0-aefefa12c7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:01.825052 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3165c-81a2-48ff-a0a0-aefefa12c7c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:01.826000 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a3165c-81a2-48ff-a0a0-aefefa12c7c1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:01.826590 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a3165c-81a2-48ff-a0a0-aefefa12c7c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:01.827787 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:01.829126 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a3165c-81a2-48ff-a0a0-aefefa12c7c1 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:01.840021 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 108/215] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 03:09:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:01.846885 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a3165c-81a2-48ff-a0a0-aefefa12c7c1 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:01.859532 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:01.870203 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea1d04-d470-4170-8b79-d2a4227b4670 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:01.912417 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf398e24-70fb-41c4-a537-0f5e3251b6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:01.913896 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf398e24-70fb-41c4-a537-0f5e3251b6df None admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:01.914354 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf398e24-70fb-41c4-a537-0f5e3251b6df None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:01.914928 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf398e24-70fb-41c4-a537-0f5e3251b6df None admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:01.915940 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf398e24-70fb-41c4-a537-0f5e3251b6df None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:01.917777 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf398e24-70fb-41c4-a537-0f5e3251b6df None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:01.946855 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 108/216] 10.222.0.50 () {60 vars in 1132 bytes} [Fri May 27 03:09:01 2022] POST /identity/v3/users => generated 297 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:09:01.994253 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a3165c-81a2-48ff-a0a0-aefefa12c7c1 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:02.034852 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552d354c-c456-4576-9ff7-e4628fc980b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:02.036144 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d354c-c456-4576-9ff7-e4628fc980b6 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:02.036423 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d354c-c456-4576-9ff7-e4628fc980b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:02.036689 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552d354c-c456-4576-9ff7-e4628fc980b6 None admin] PATH_INFO: `/v3/users/10114c80d86f43bdb74ff9a5ffebcdd9` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:02.053258 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552d354c-c456-4576-9ff7-e4628fc980b6 None admin] RBAC: Authorizing `identity:get_user(user_id=10114c80d86f43bdb74ff9a5ffebcdd9)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:02.054912 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552d354c-c456-4576-9ff7-e4628fc980b6 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:02.057022 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 109/217] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:09:01 2022] GET /identity/v3/users/10114c80d86f43bdb74ff9a5ffebcdd9 => generated 297 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:02.062716 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf398e24-70fb-41c4-a537-0f5e3251b6df None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:02.112561 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fdc75-3266-4df3-be9a-ea5295fc7f96 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:02.113223 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fdc75-3266-4df3-be9a-ea5295fc7f96 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:02.113561 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fdc75-3266-4df3-be9a-ea5295fc7f96 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:02.115195 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 110/218] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:09:02.121588 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335409f6-880b-41bb-bbc7-e48487224815 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:02.122211 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335409f6-880b-41bb-bbc7-e48487224815 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:02.122683 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335409f6-880b-41bb-bbc7-e48487224815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:02.144549 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-335409f6-880b-41bb-bbc7-e48487224815 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:02.146394 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-335409f6-880b-41bb-bbc7-e48487224815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:02.174588 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf398e24-70fb-41c4-a537-0f5e3251b6df None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:02.202495 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:02.218684 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90a4de5-ff75-4787-8df7-958e8eec6916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:02.219941 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:02.221048 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a4de5-ff75-4787-8df7-958e8eec6916 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:02.221591 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a4de5-ff75-4787-8df7-958e8eec6916 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:02.222001 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90a4de5-ff75-4787-8df7-958e8eec6916 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:02.229870 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 111/219] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:09:02.254034 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335409f6-880b-41bb-bbc7-e48487224815 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:02.297695 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90a4de5-ff75-4787-8df7-958e8eec6916 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:02.298728 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90a4de5-ff75-4787-8df7-958e8eec6916 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:02.397214 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335409f6-880b-41bb-bbc7-e48487224815 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:02.434991 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 109/220] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:09:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:09:02.437948 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f9397-852b-4f68-b79e-e60568a5ba5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:02.438757 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f9397-852b-4f68-b79e-e60568a5ba5d None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:02.439075 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f9397-852b-4f68-b79e-e60568a5ba5d None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:02.439353 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f9397-852b-4f68-b79e-e60568a5ba5d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:02.440887 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90a4de5-ff75-4787-8df7-958e8eec6916 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:02.451002 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31f9397-852b-4f68-b79e-e60568a5ba5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:02.459210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-d31f9397-852b-4f68-b79e-e60568a5ba5d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:09:02.459210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:02.459210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:02.459210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:02.459210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:02.459210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:02.459210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:02.459210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:02.459210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:02.459210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:02.459210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:02.459210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:02.459210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:02.459210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:02.459210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:02.459210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:02.460566 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:02.460566 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:02.460566 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:02.460566 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:02.460566 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:02.460566 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:02.460566 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:02.460566 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:02.460566 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:02.460566 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:02.460566 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:02.460566 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:02.460566 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:02.460566 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:02.460566 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:02.460566 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:02.461452 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:02.461452 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:02.461452 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:02.461452 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:02.461452 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:02.461452 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:02.461452 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:02.461452 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:02.461452 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:02.461452 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:02.461452 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:09:02.461452 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:02.464118 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 112/221] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 03:09:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:02.470557 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:02.471691 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f9397-852b-4f68-b79e-e60568a5ba5d None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:02.547313 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:02.579060 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90a4de5-ff75-4787-8df7-958e8eec6916 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:02.598856 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f9397-852b-4f68-b79e-e60568a5ba5d None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:02.627087 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16f779-75fb-4b12-9b69-9d9a6107b41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:02.628197 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16f779-75fb-4b12-9b69-9d9a6107b41d None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:02.628473 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16f779-75fb-4b12-9b69-9d9a6107b41d None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:02.628746 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16f779-75fb-4b12-9b69-9d9a6107b41d None admin] PATH_INFO: `/v3/projects/5e327e3411214877a9f427330ef40dc5` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:02.630656 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d16f779-75fb-4b12-9b69-9d9a6107b41d None admin] RBAC: Authorizing `identity:get_project(project_id=5e327e3411214877a9f427330ef40dc5)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:02.633655 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d16f779-75fb-4b12-9b69-9d9a6107b41d None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:02.638892 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 110/222] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:09:02 2022] GET /identity/v3/projects/5e327e3411214877a9f427330ef40dc5 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:02.642111 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16f779-75fb-4b12-9b69-9d9a6107b41d None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:02.642855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a35279-5017-4dc5-938e-32232caf94d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:02.643752 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a35279-5017-4dc5-938e-32232caf94d7 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:02.644277 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a35279-5017-4dc5-938e-32232caf94d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:02.644711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a35279-5017-4dc5-938e-32232caf94d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:02.645763 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a35279-5017-4dc5-938e-32232caf94d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:02.646245 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a35279-5017-4dc5-938e-32232caf94d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:02.648006 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a35279-5017-4dc5-938e-32232caf94d7 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:02.658507 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 113/223] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:09:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:02.664688 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a35279-5017-4dc5-938e-32232caf94d7 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:02.791150 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:02.805483 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16f779-75fb-4b12-9b69-9d9a6107b41d None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:02.807603 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a35279-5017-4dc5-938e-32232caf94d7 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:02.833387 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:02.864238 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b2097-414d-4aa7-83fe-84e258ffe123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:02.865749 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b2097-414d-4aa7-83fe-84e258ffe123 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:02.866719 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b2097-414d-4aa7-83fe-84e258ffe123 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:02.867563 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc256ee-1057-4b36-839b-fa0d97c871d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:02.868400 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc256ee-1057-4b36-839b-fa0d97c871d7 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:02.868838 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b2097-414d-4aa7-83fe-84e258ffe123 None admin] PATH_INFO: `/v3/users/8b1afed4cb0342a783c95efb73a8b869` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:02.869548 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc256ee-1057-4b36-839b-fa0d97c871d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:02.870371 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc256ee-1057-4b36-839b-fa0d97c871d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:02.871162 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9b2097-414d-4aa7-83fe-84e258ffe123 None admin] RBAC: Authorizing `identity:get_user(user_id=8b1afed4cb0342a783c95efb73a8b869)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:02.872159 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc256ee-1057-4b36-839b-fa0d97c871d7 None admin] RBAC: Adding query filter params (role.id=0129dd45ff5449f9957e8efdd01a55ed, scope.project.id=5e327e3411214877a9f427330ef40dc5, user.id=10114c80d86f43bdb74ff9a5ffebcdd9) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:02.873825 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9b2097-414d-4aa7-83fe-84e258ffe123 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:02.874402 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc256ee-1057-4b36-839b-fa0d97c871d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:02.876013 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc256ee-1057-4b36-839b-fa0d97c871d7 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:02.877856 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 114/224] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:09:02 2022] GET /identity/v3/users/8b1afed4cb0342a783c95efb73a8b869 => generated 283 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:02.886573 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b2097-414d-4aa7-83fe-84e258ffe123 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:02.898592 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 111/225] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:09:02 2022] GET /identity/v3/role_assignments?user.id=10114c80d86f43bdb74ff9a5ffebcdd9&scope.project.id=5e327e3411214877a9f427330ef40dc5&role.id=0129dd45ff5449f9957e8efdd01a55ed => generated 259 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:03.049665 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b2097-414d-4aa7-83fe-84e258ffe123 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:03.105802 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5ba96-499b-4b94-b3ce-2d5aeb01e9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:03.107026 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5ba96-499b-4b94-b3ce-2d5aeb01e9a8 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:03.107477 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5ba96-499b-4b94-b3ce-2d5aeb01e9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:03.107917 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5ba96-499b-4b94-b3ce-2d5aeb01e9a8 None admin] PATH_INFO: `/v3/projects/5e327e3411214877a9f427330ef40dc5` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:03.109518 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe5ba96-499b-4b94-b3ce-2d5aeb01e9a8 None admin] RBAC: Authorizing `identity:get_project(project_id=5e327e3411214877a9f427330ef40dc5)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:03.111184 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe5ba96-499b-4b94-b3ce-2d5aeb01e9a8 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:03.113097 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 115/226] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:09:02 2022] GET /identity/v3/projects/5e327e3411214877a9f427330ef40dc5 => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:03.119194 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc256ee-1057-4b36-839b-fa0d97c871d7 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:03.175660 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e7624-72d5-42d4-aeea-411a06504786 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:03.176252 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e7624-72d5-42d4-aeea-411a06504786 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:03.176716 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e7624-72d5-42d4-aeea-411a06504786 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:03.178313 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 116/227] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:09:03.183996 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f0cc8-284b-4e41-8b0c-aac9a4dc21ed None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:03.184290 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f0cc8-284b-4e41-8b0c-aac9a4dc21ed None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:03.184571 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f0cc8-284b-4e41-8b0c-aac9a4dc21ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:03.213791 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-cb2f0cc8-284b-4e41-8b0c-aac9a4dc21ed None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:03.215087 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-cb2f0cc8-284b-4e41-8b0c-aac9a4dc21ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:03.257055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc256ee-1057-4b36-839b-fa0d97c871d7 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:03.304984 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad11c660-7ee4-482c-a07e-a0ddd03e9cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:03.304984 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad11c660-7ee4-482c-a07e-a0ddd03e9cd9 None admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:03.304984 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad11c660-7ee4-482c-a07e-a0ddd03e9cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:03.304984 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad11c660-7ee4-482c-a07e-a0ddd03e9cd9 None admin] PATH_INFO: `/v3/projects/5e327e3411214877a9f427330ef40dc5/users/8b1afed4cb0342a783c95efb73a8b869/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:03.309466 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad11c660-7ee4-482c-a07e-a0ddd03e9cd9 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e327e3411214877a9f427330ef40dc5, user_id=8b1afed4cb0342a783c95efb73a8b869, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:03.313145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad11c660-7ee4-482c-a07e-a0ddd03e9cd9 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:03.323410 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:03.339939 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 112/228] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 03:09:03 2022] PUT /identity/v3/projects/5e327e3411214877a9f427330ef40dc5/users/8b1afed4cb0342a783c95efb73a8b869/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:09:03.355183 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 117/229] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:09:03.385298 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad11c660-7ee4-482c-a07e-a0ddd03e9cd9 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:03.411096 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:03.531612 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad11c660-7ee4-482c-a07e-a0ddd03e9cd9 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:03.572781 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e61c3f3-550d-425a-bf9f-cf6c3e21e8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:03.573987 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e61c3f3-550d-425a-bf9f-cf6c3e21e8b7 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:03.574466 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e61c3f3-550d-425a-bf9f-cf6c3e21e8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:03.574924 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e61c3f3-550d-425a-bf9f-cf6c3e21e8b7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:03.586292 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e61c3f3-550d-425a-bf9f-cf6c3e21e8b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:03.594904 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-5e61c3f3-550d-425a-bf9f-cf6c3e21e8b7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:09:03.594904 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:03.594904 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:03.594904 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:03.594904 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:03.594904 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:03.594904 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:03.594904 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:03.594904 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:03.594904 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:03.594904 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:03.594904 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:03.594904 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:03.594904 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:03.594904 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:03.594904 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:03.595748 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:03.595748 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:03.595748 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:03.595748 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:03.595748 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:03.595748 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:03.595748 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:03.595748 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:03.595748 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:03.595748 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:03.595748 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:03.595748 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:03.595748 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:03.595748 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:03.595748 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:03.595748 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:03.596519 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:03.596519 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:03.596519 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:03.596519 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:03.596519 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:03.596519 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:03.596519 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:03.596519 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:03.596519 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:03.596519 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:03.596519 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:09:03.596519 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:03.599228 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 113/230] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 03:09:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:03.609035 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f0cc8-284b-4e41-8b0c-aac9a4dc21ed None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:03.720321 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f0cc8-284b-4e41-8b0c-aac9a4dc21ed None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:03.770715 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7e06c-f313-4f06-807f-9804978cf1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:03.771389 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7e06c-f313-4f06-807f-9804978cf1bd None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:03.771668 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7e06c-f313-4f06-807f-9804978cf1bd None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:03.771941 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7e06c-f313-4f06-807f-9804978cf1bd None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:03.772726 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab7e06c-f313-4f06-807f-9804978cf1bd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:03.773090 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab7e06c-f313-4f06-807f-9804978cf1bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:03.774539 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab7e06c-f313-4f06-807f-9804978cf1bd None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:03.783502 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 118/231] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:09:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:03.789557 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e61c3f3-550d-425a-bf9f-cf6c3e21e8b7 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:03.886371 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:03.915286 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e61c3f3-550d-425a-bf9f-cf6c3e21e8b7 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:03.959138 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ede675-8dca-4347-a4df-93ba94d9121b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:03.960300 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ede675-8dca-4347-a4df-93ba94d9121b None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:03.960640 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ede675-8dca-4347-a4df-93ba94d9121b None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:03.960863 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ede675-8dca-4347-a4df-93ba94d9121b None admin] PATH_INFO: `/v3/users/8b1afed4cb0342a783c95efb73a8b869` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:03.965694 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ede675-8dca-4347-a4df-93ba94d9121b None admin] RBAC: Authorizing `identity:get_user(user_id=8b1afed4cb0342a783c95efb73a8b869)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:03.967187 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ede675-8dca-4347-a4df-93ba94d9121b None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:03.969283 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 114/232] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:09:03 2022] GET /identity/v3/users/8b1afed4cb0342a783c95efb73a8b869 => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:03.974592 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7e06c-f313-4f06-807f-9804978cf1bd None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:04.035246 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:04.105338 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7e06c-f313-4f06-807f-9804978cf1bd None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:04.144306 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a4f3e-f0aa-44df-a698-011866b73ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:04.145209 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a4f3e-f0aa-44df-a698-011866b73ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:04.145485 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a4f3e-f0aa-44df-a698-011866b73ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:04.145754 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a4f3e-f0aa-44df-a698-011866b73ce2 None admin] PATH_INFO: `/v3/projects/5e327e3411214877a9f427330ef40dc5` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:04.146940 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a4f3e-f0aa-44df-a698-011866b73ce2 None admin] RBAC: Authorizing `identity:get_project(project_id=5e327e3411214877a9f427330ef40dc5)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:04.148644 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057a4f3e-f0aa-44df-a698-011866b73ce2 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:04.150372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 119/233] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:09:03 2022] GET /identity/v3/projects/5e327e3411214877a9f427330ef40dc5 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:04.156106 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ede675-8dca-4347-a4df-93ba94d9121b None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:04.302477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ede675-8dca-4347-a4df-93ba94d9121b None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:04.340216 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af02d39-2f73-4b79-b421-517a139f582c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:04.341149 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af02d39-2f73-4b79-b421-517a139f582c None admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:04.341426 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af02d39-2f73-4b79-b421-517a139f582c None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:04.341694 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af02d39-2f73-4b79-b421-517a139f582c None admin] PATH_INFO: `/v3/projects/5e327e3411214877a9f427330ef40dc5/users/8b1afed4cb0342a783c95efb73a8b869/roles/9e63b73b3ce24bb3ad214ecdb6132696` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:04.343899 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af02d39-2f73-4b79-b421-517a139f582c None admin] RBAC: Authorizing `identity:create_grant(project_id=5e327e3411214877a9f427330ef40dc5, user_id=8b1afed4cb0342a783c95efb73a8b869, role_id=9e63b73b3ce24bb3ad214ecdb6132696)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:04.345591 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af02d39-2f73-4b79-b421-517a139f582c None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:04.362945 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 115/234] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 03:09:04 2022] PUT /identity/v3/projects/5e327e3411214877a9f427330ef40dc5/users/8b1afed4cb0342a783c95efb73a8b869/roles/9e63b73b3ce24bb3ad214ecdb6132696 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:09:04.848912 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f386f8-c644-4188-bcda-ade6214491df None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:04.849484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f386f8-c644-4188-bcda-ade6214491df None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:04.849484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f386f8-c644-4188-bcda-ade6214491df None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:04.851327 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 120/235] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:04.857511 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290a998-62cc-4ddd-9f1b-8ae17ab21743 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:04.858027 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290a998-62cc-4ddd-9f1b-8ae17ab21743 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:04.858128 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290a998-62cc-4ddd-9f1b-8ae17ab21743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:04.865056 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:04.877681 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-2290a998-62cc-4ddd-9f1b-8ae17ab21743 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:04.878904 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-2290a998-62cc-4ddd-9f1b-8ae17ab21743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:04.970442 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 116/236] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:05.006862 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f386f8-c644-4188-bcda-ade6214491df None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:05.087345 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858120de-6914-46e1-abf9-3eb13fe247bb None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:05.087724 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858120de-6914-46e1-abf9-3eb13fe247bb None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:05.088005 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858120de-6914-46e1-abf9-3eb13fe247bb None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:05.089617 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 117/237] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:05.095277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe511e5-fffe-4caf-b19d-275fcbfe3d5f None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:05.095596 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe511e5-fffe-4caf-b19d-275fcbfe3d5f None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:05.095944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe511e5-fffe-4caf-b19d-275fcbfe3d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:05.124246 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-afe511e5-fffe-4caf-b19d-275fcbfe3d5f None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:05.125385 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-afe511e5-fffe-4caf-b19d-275fcbfe3d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:05.133720 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f386f8-c644-4188-bcda-ade6214491df None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:05.157313 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:05.182612 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4cf500-a166-43f1-a185-ec537312e91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:05.183423 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4cf500-a166-43f1-a185-ec537312e91e None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:05.183707 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4cf500-a166-43f1-a185-ec537312e91e None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:05.183983 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4cf500-a166-43f1-a185-ec537312e91e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:05.195649 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4cf500-a166-43f1-a185-ec537312e91e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:05.203720 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-de4cf500-a166-43f1-a185-ec537312e91e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:09:05.203720 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:05.203720 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:05.203720 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:05.203720 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:05.203720 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:05.203720 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:05.203720 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:05.203720 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:05.203720 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:05.203720 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:05.203720 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:05.203720 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:05.203720 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:05.203720 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:05.203720 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:05.204569 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:05.204569 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:05.204569 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:05.204569 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:05.204569 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:05.204569 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:05.204569 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:05.204569 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:05.204569 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:05.204569 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:05.204569 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:05.204569 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:05.204569 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:05.204569 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:05.204569 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:05.204569 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:05.205373 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:05.205373 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:05.205373 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:05.205373 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:05.205373 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:05.205373 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:05.205373 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:05.205373 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:05.205373 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:05.205373 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:05.205373 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:09:05.205373 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:05.209059 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 121/238] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:09:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:05.211178 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4cf500-a166-43f1-a185-ec537312e91e None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:05.230079 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 118/239] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:05.279730 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe511e5-fffe-4caf-b19d-275fcbfe3d5f None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:05.315107 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4cf500-a166-43f1-a185-ec537312e91e None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:05.350509 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46945cda-7e09-4ae4-8c11-998902e0d034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:05.351253 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46945cda-7e09-4ae4-8c11-998902e0d034 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:05.351471 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46945cda-7e09-4ae4-8c11-998902e0d034 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:05.351749 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46945cda-7e09-4ae4-8c11-998902e0d034 None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:05.352501 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46945cda-7e09-4ae4-8c11-998902e0d034 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:05.352922 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46945cda-7e09-4ae4-8c11-998902e0d034 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:05.354366 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46945cda-7e09-4ae4-8c11-998902e0d034 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:05.367353 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 122/240] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 03:09:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:05.369108 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46945cda-7e09-4ae4-8c11-998902e0d034 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:05.404679 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:05.409985 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:05.476329 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46945cda-7e09-4ae4-8c11-998902e0d034 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:05.487697 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe511e5-fffe-4caf-b19d-275fcbfe3d5f None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:05.512493 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f712434f-ee54-4a2d-9feb-6d5bdb265379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:05.513513 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f712434f-ee54-4a2d-9feb-6d5bdb265379 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:05.513793 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f712434f-ee54-4a2d-9feb-6d5bdb265379 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:05.514065 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f712434f-ee54-4a2d-9feb-6d5bdb265379 None admin] PATH_INFO: `/v3/users/81e478bc272140e79c7ca72302862581` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:05.528660 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f712434f-ee54-4a2d-9feb-6d5bdb265379 None admin] RBAC: Authorizing `identity:get_user(user_id=81e478bc272140e79c7ca72302862581)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:05.530189 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f712434f-ee54-4a2d-9feb-6d5bdb265379 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:05.532109 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 123/241] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:09:05 2022] GET /identity/v3/users/81e478bc272140e79c7ca72302862581 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:05.537926 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f712434f-ee54-4a2d-9feb-6d5bdb265379 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:05.557767 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fe73a8-7a49-43f6-a5cf-cb0759ad28ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:05.558590 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe73a8-7a49-43f6-a5cf-cb0759ad28ef None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:05.558869 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe73a8-7a49-43f6-a5cf-cb0759ad28ef None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:05.559193 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fe73a8-7a49-43f6-a5cf-cb0759ad28ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:05.560506 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fe73a8-7a49-43f6-a5cf-cb0759ad28ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:05.562018 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fe73a8-7a49-43f6-a5cf-cb0759ad28ef None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:05.563855 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 119/242] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:05.570453 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fe73a8-7a49-43f6-a5cf-cb0759ad28ef None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:05.633879 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:05.642288 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f712434f-ee54-4a2d-9feb-6d5bdb265379 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:05.694176 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298a299-ddf1-4cc8-a52b-59964043aa6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:05.694615 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298a299-ddf1-4cc8-a52b-59964043aa6b None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:05.694903 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298a299-ddf1-4cc8-a52b-59964043aa6b None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:05.695176 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298a299-ddf1-4cc8-a52b-59964043aa6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:05.726599 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fe73a8-7a49-43f6-a5cf-cb0759ad28ef None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:05.772816 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f298a299-ddf1-4cc8-a52b-59964043aa6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:05.774517 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f298a299-ddf1-4cc8-a52b-59964043aa6b None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:05.834416 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:05.858561 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6381448d-3d16-4eb0-ba4a-4d7006ccf6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:05.859242 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381448d-3d16-4eb0-ba4a-4d7006ccf6d0 None admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:05.859531 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381448d-3d16-4eb0-ba4a-4d7006ccf6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:05.859802 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381448d-3d16-4eb0-ba4a-4d7006ccf6d0 None admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:05.860524 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6381448d-3d16-4eb0-ba4a-4d7006ccf6d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:05.862071 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6381448d-3d16-4eb0-ba4a-4d7006ccf6d0 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:05.885330 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:05.889476 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 120/243] 10.222.0.50 () {60 vars in 1132 bytes} [Fri May 27 03:09:05 2022] POST /identity/v3/users => generated 293 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:09:05.891421 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f20202-1421-4e7b-9600-4a0f3e710829 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:05.892022 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f20202-1421-4e7b-9600-4a0f3e710829 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:05.892300 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f20202-1421-4e7b-9600-4a0f3e710829 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:05.893626 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 124/244] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:09:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 03:09:05.894160 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 121/245] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:05.900285 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202069d8-7e24-4049-b034-695f8a984187 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:05.900571 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202069d8-7e24-4049-b034-695f8a984187 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:05.900846 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202069d8-7e24-4049-b034-695f8a984187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:05.901970 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f20202-1421-4e7b-9600-4a0f3e710829 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:05.922922 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-202069d8-7e24-4049-b034-695f8a984187 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:05.924131 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-202069d8-7e24-4049-b034-695f8a984187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:06.045140 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 125/246] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:06.059016 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f20202-1421-4e7b-9600-4a0f3e710829 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:06.079456 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202069d8-7e24-4049-b034-695f8a984187 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:06.116256 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a68363-ffd3-47d7-a674-7dbe2b5f51e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:06.117210 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a68363-ffd3-47d7-a674-7dbe2b5f51e2 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:06.117489 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a68363-ffd3-47d7-a674-7dbe2b5f51e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:06.117762 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a68363-ffd3-47d7-a674-7dbe2b5f51e2 None admin] PATH_INFO: `/v3/projects/5e327e3411214877a9f427330ef40dc5` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:06.118969 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a68363-ffd3-47d7-a674-7dbe2b5f51e2 None admin] RBAC: Authorizing `identity:get_project(project_id=5e327e3411214877a9f427330ef40dc5)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:06.120761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a68363-ffd3-47d7-a674-7dbe2b5f51e2 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:06.123705 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 122/247] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:09:05 2022] GET /identity/v3/projects/5e327e3411214877a9f427330ef40dc5 => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:06.137324 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a68363-ffd3-47d7-a674-7dbe2b5f51e2 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:06.213619 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202069d8-7e24-4049-b034-695f8a984187 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:06.217108 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:06.249352 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:06.263822 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd51893-97da-42ac-b12e-3d0b46bb1edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:06.264638 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd51893-97da-42ac-b12e-3d0b46bb1edc None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:06.266627 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd51893-97da-42ac-b12e-3d0b46bb1edc None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:06.266627 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd51893-97da-42ac-b12e-3d0b46bb1edc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:06.279374 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd51893-97da-42ac-b12e-3d0b46bb1edc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:06.285813 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a68363-ffd3-47d7-a674-7dbe2b5f51e2 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:06.287348 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-7fd51893-97da-42ac-b12e-3d0b46bb1edc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:09:06.287348 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:06.287348 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:06.287348 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:06.287348 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:06.287348 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:06.287348 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:06.287348 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:06.287348 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:06.287348 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:06.287348 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:06.287348 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:06.287348 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:06.287348 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:06.287348 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:06.287348 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:06.288313 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:06.288313 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:06.288313 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:06.288313 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:06.288313 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:06.288313 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:06.288313 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:06.288313 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:06.288313 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:06.288313 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:06.288313 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:06.288313 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:06.288313 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:06.288313 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:06.288313 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:06.288313 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:06.289119 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:06.289119 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:06.289119 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:06.289119 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:06.289119 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:06.289119 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:06.289119 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:06.289119 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:06.289119 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:06.289119 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:06.289119 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:09:06.289119 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:06.291289 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 126/248] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:09:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:09:06.298916 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd51893-97da-42ac-b12e-3d0b46bb1edc None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:06.334419 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fc9bc5-fa3c-4a8e-b5ce-b8ab6355c2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:06.335362 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc9bc5-fa3c-4a8e-b5ce-b8ab6355c2f1 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:06.335784 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc9bc5-fa3c-4a8e-b5ce-b8ab6355c2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:06.336223 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fc9bc5-fa3c-4a8e-b5ce-b8ab6355c2f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:06.337270 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fc9bc5-fa3c-4a8e-b5ce-b8ab6355c2f1 None admin] RBAC: Adding query filter params (role.id=6479235705f74061aff23df5c9f89e33, scope.project.id=5e327e3411214877a9f427330ef40dc5, user.id=81e478bc272140e79c7ca72302862581) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:06.337723 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fc9bc5-fa3c-4a8e-b5ce-b8ab6355c2f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:06.339421 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fc9bc5-fa3c-4a8e-b5ce-b8ab6355c2f1 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:06.346099 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 123/249] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:09:06 2022] GET /identity/v3/role_assignments?user.id=81e478bc272140e79c7ca72302862581&scope.project.id=5e327e3411214877a9f427330ef40dc5&role.id=6479235705f74061aff23df5c9f89e33 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:06.416399 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e94b34-4c41-425c-9089-39e7fc3c35b8 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:06.416916 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e94b34-4c41-425c-9089-39e7fc3c35b8 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:06.417428 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e94b34-4c41-425c-9089-39e7fc3c35b8 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:06.425142 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd51893-97da-42ac-b12e-3d0b46bb1edc None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:06.425812 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 124/250] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:06 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:06.432117 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd3f670-a36d-4371-a475-b35d36006cab None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:06.432424 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd3f670-a36d-4371-a475-b35d36006cab None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:06.432660 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd3f670-a36d-4371-a475-b35d36006cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:06.455716 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-8fd3f670-a36d-4371-a475-b35d36006cab None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:06.456932 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-8fd3f670-a36d-4371-a475-b35d36006cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:06.463952 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a33e09-6d58-4474-a027-c6b1d6dcf794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:06.464646 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a33e09-6d58-4474-a027-c6b1d6dcf794 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:06.464924 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a33e09-6d58-4474-a027-c6b1d6dcf794 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:06.465436 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a33e09-6d58-4474-a027-c6b1d6dcf794 None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:06.466199 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a33e09-6d58-4474-a027-c6b1d6dcf794 None admin] RBAC: Adding query filter params (name=member) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:06.466544 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a33e09-6d58-4474-a027-c6b1d6dcf794 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:06.468028 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a33e09-6d58-4474-a027-c6b1d6dcf794 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:06.478825 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 127/251] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 03:09:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:06.484512 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a33e09-6d58-4474-a027-c6b1d6dcf794 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:06.541132 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 125/252] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:09:06.542602 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:06.566386 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd3f670-a36d-4371-a475-b35d36006cab None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:06.606244 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a33e09-6d58-4474-a027-c6b1d6dcf794 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:06.629587 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:06.645432 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f24578-2a6d-4f16-b006-b759b69f8c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:06.646330 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f24578-2a6d-4f16-b006-b759b69f8c2d None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:06.646613 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f24578-2a6d-4f16-b006-b759b69f8c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:06.646985 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f24578-2a6d-4f16-b006-b759b69f8c2d None admin] PATH_INFO: `/v3/users/10114c80d86f43bdb74ff9a5ffebcdd9` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:06.648594 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f24578-2a6d-4f16-b006-b759b69f8c2d None admin] RBAC: Authorizing `identity:get_user(user_id=10114c80d86f43bdb74ff9a5ffebcdd9)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:06.650151 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f24578-2a6d-4f16-b006-b759b69f8c2d None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:06.652123 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 128/253] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:09:06 2022] GET /identity/v3/users/10114c80d86f43bdb74ff9a5ffebcdd9 => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:06.658209 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f24578-2a6d-4f16-b006-b759b69f8c2d None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:06.684601 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd3f670-a36d-4371-a475-b35d36006cab None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:06.719953 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980744d-ed0c-4f70-bd78-0c985497e131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:06.720823 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980744d-ed0c-4f70-bd78-0c985497e131 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:06.721134 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980744d-ed0c-4f70-bd78-0c985497e131 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:06.721407 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980744d-ed0c-4f70-bd78-0c985497e131 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:06.734677 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3980744d-ed0c-4f70-bd78-0c985497e131 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:06.746761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-3980744d-ed0c-4f70-bd78-0c985497e131 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:09:06.746761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:06.746761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:06.746761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:06.746761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:06.746761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:06.746761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:06.746761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:06.746761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:06.746761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:06.746761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:06.746761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:06.746761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:06.746761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:06.746761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:06.746761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:06.747695 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:06.747695 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:06.747695 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:06.747695 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:06.747695 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:06.747695 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:06.747695 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:06.747695 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:06.747695 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:06.747695 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:06.747695 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:06.747695 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:06.747695 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:06.747695 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:06.747695 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:06.747695 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:06.748486 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:06.748486 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:06.748486 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:06.748486 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:06.748486 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:06.748486 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:06.748486 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:06.748486 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:06.748486 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:06.748486 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:06.748486 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:09:06.748486 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:06.748486 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 126/254] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 03:09:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:09:06.755506 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980744d-ed0c-4f70-bd78-0c985497e131 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:06.859786 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f24578-2a6d-4f16-b006-b759b69f8c2d None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:06.873807 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:06.908150 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6782a01-0db8-4bba-8114-d59c1c2c4b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:06.909073 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6782a01-0db8-4bba-8114-d59c1c2c4b6d None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:06.913544 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6782a01-0db8-4bba-8114-d59c1c2c4b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:06.913947 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6782a01-0db8-4bba-8114-d59c1c2c4b6d None admin] PATH_INFO: `/v3/projects/5e327e3411214877a9f427330ef40dc5` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:06.915268 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6782a01-0db8-4bba-8114-d59c1c2c4b6d None admin] RBAC: Authorizing `identity:get_project(project_id=5e327e3411214877a9f427330ef40dc5)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:06.916832 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6782a01-0db8-4bba-8114-d59c1c2c4b6d None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:06.918913 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 129/255] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:09:06 2022] GET /identity/v3/projects/5e327e3411214877a9f427330ef40dc5 => generated 302 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:06.925076 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6782a01-0db8-4bba-8114-d59c1c2c4b6d None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:06.927070 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980744d-ed0c-4f70-bd78-0c985497e131 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:06.949184 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:07.060850 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6782a01-0db8-4bba-8114-d59c1c2c4b6d None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:07.078299 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82670ee-451f-45ae-b54d-1d645e213fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:07.078993 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82670ee-451f-45ae-b54d-1d645e213fc5 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:07.079273 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82670ee-451f-45ae-b54d-1d645e213fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:07.079539 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82670ee-451f-45ae-b54d-1d645e213fc5 None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:07.080303 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82670ee-451f-45ae-b54d-1d645e213fc5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:07.080605 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82670ee-451f-45ae-b54d-1d645e213fc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:07.082033 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82670ee-451f-45ae-b54d-1d645e213fc5 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:07.100497 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 127/256] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:09:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:07.105954 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82670ee-451f-45ae-b54d-1d645e213fc5 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:07.137332 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4cbb77-8f14-4514-ba34-e373961b8959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:07.138174 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4cbb77-8f14-4514-ba34-e373961b8959 None admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:07.138447 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4cbb77-8f14-4514-ba34-e373961b8959 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:07.138721 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4cbb77-8f14-4514-ba34-e373961b8959 None admin] PATH_INFO: `/v3/projects/5e327e3411214877a9f427330ef40dc5/users/10114c80d86f43bdb74ff9a5ffebcdd9/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:07.140706 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4cbb77-8f14-4514-ba34-e373961b8959 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e327e3411214877a9f427330ef40dc5, user_id=10114c80d86f43bdb74ff9a5ffebcdd9, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:07.142745 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4cbb77-8f14-4514-ba34-e373961b8959 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:07.178716 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 130/257] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 03:09:06 2022] PUT /identity/v3/projects/5e327e3411214877a9f427330ef40dc5/users/10114c80d86f43bdb74ff9a5ffebcdd9/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:09:07.282231 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82670ee-451f-45ae-b54d-1d645e213fc5 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:07.332840 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc9a14-5787-4397-ac76-e9acca45340a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:07.333772 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc9a14-5787-4397-ac76-e9acca45340a None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:07.334046 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc9a14-5787-4397-ac76-e9acca45340a None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:07.334409 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc9a14-5787-4397-ac76-e9acca45340a None admin] PATH_INFO: `/v3/users/8b1afed4cb0342a783c95efb73a8b869` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:07.335670 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc9a14-5787-4397-ac76-e9acca45340a None admin] RBAC: Authorizing `identity:get_user(user_id=8b1afed4cb0342a783c95efb73a8b869)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:07.337213 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc9a14-5787-4397-ac76-e9acca45340a None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:07.338868 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 128/258] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:09:07 2022] GET /identity/v3/users/8b1afed4cb0342a783c95efb73a8b869 => generated 283 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:07.344529 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4cbb77-8f14-4514-ba34-e373961b8959 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:07.390046 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:07.452938 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4cbb77-8f14-4514-ba34-e373961b8959 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:07.488402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8aeeee-73c2-4ede-acfd-71f35604ce9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:07.489142 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8aeeee-73c2-4ede-acfd-71f35604ce9f None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:07.489737 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8aeeee-73c2-4ede-acfd-71f35604ce9f None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:07.490050 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8aeeee-73c2-4ede-acfd-71f35604ce9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:07.562476 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8aeeee-73c2-4ede-acfd-71f35604ce9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:07.563962 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8aeeee-73c2-4ede-acfd-71f35604ce9f None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:07.574658 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39549f6a-0c00-452b-8886-d38ea5b43f12 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:07.574967 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39549f6a-0c00-452b-8886-d38ea5b43f12 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:07.575203 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39549f6a-0c00-452b-8886-d38ea5b43f12 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:07.576764 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 129/259] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:09:07.590829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2f8fe-a3c1-4ed6-b805-078590ee8187 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:07.591133 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2f8fe-a3c1-4ed6-b805-078590ee8187 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:07.591372 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2f8fe-a3c1-4ed6-b805-078590ee8187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:07.617258 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-1ba2f8fe-a3c1-4ed6-b805-078590ee8187 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:07.620089 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-1ba2f8fe-a3c1-4ed6-b805-078590ee8187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:07.640329 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:07.697336 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:07.739467 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 130/260] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:09:07.750115 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 131/261] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:09:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:09:07.755740 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba2f8fe-a3c1-4ed6-b805-078590ee8187 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:07.779272 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8aeeee-73c2-4ede-acfd-71f35604ce9f None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:07.943459 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba2f8fe-a3c1-4ed6-b805-078590ee8187 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:07.966071 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8aeeee-73c2-4ede-acfd-71f35604ce9f None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:08.001129 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad905791-e095-48c3-9f39-6240cbfcd7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:08.001950 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad905791-e095-48c3-9f39-6240cbfcd7c2 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:08.002241 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad905791-e095-48c3-9f39-6240cbfcd7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:08.002493 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad905791-e095-48c3-9f39-6240cbfcd7c2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:08.012457 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:08.013749 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad905791-e095-48c3-9f39-6240cbfcd7c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:08.021658 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358032a2-5b20-4ca1-8a1c-b2697b7a00d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:08.022542 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-ad905791-e095-48c3-9f39-6240cbfcd7c2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:09:08.022542 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:08.022542 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:08.022542 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:08.022542 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:08.022542 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:08.022542 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:08.022542 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:08.022542 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:08.022542 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:08.022542 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:08.022542 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:08.022542 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:08.022542 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:08.022542 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:08.022542 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:08.023586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:08.023586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:08.023586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:08.023586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:08.023586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:08.023586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:08.023586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:08.023586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:08.023586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:08.023586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:08.023586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:08.023586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:08.023586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:08.023586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:08.023586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:08.023586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:08.024579 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:08.024579 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:08.024579 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:08.024579 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:08.024579 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:08.024579 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:08.024579 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:08.024579 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:08.024579 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:08.024579 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:08.024579 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:09:08.024579 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:08.025926 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358032a2-5b20-4ca1-8a1c-b2697b7a00d2 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:08.026044 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 132/262] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 03:09:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:09:08.026505 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358032a2-5b20-4ca1-8a1c-b2697b7a00d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:08.026933 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358032a2-5b20-4ca1-8a1c-b2697b7a00d2 None admin] PATH_INFO: `/v3/projects/5e327e3411214877a9f427330ef40dc5` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:08.029842 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358032a2-5b20-4ca1-8a1c-b2697b7a00d2 None admin] RBAC: Authorizing `identity:get_project(project_id=5e327e3411214877a9f427330ef40dc5)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:08.032290 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358032a2-5b20-4ca1-8a1c-b2697b7a00d2 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:08.033362 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad905791-e095-48c3-9f39-6240cbfcd7c2 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:08.038683 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 131/263] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:09:07 2022] GET /identity/v3/projects/5e327e3411214877a9f427330ef40dc5 => generated 302 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:08.049143 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358032a2-5b20-4ca1-8a1c-b2697b7a00d2 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:08.122327 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:08.199278 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad905791-e095-48c3-9f39-6240cbfcd7c2 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:08.233024 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358032a2-5b20-4ca1-8a1c-b2697b7a00d2 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:08.259717 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b435da-1ac5-4c0f-94af-5bb194a2a18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:08.260741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b435da-1ac5-4c0f-94af-5bb194a2a18f None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:08.261253 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b435da-1ac5-4c0f-94af-5bb194a2a18f None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:08.261710 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b435da-1ac5-4c0f-94af-5bb194a2a18f None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:08.262697 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b435da-1ac5-4c0f-94af-5bb194a2a18f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:08.265279 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b435da-1ac5-4c0f-94af-5bb194a2a18f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:08.266887 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b435da-1ac5-4c0f-94af-5bb194a2a18f None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:08.278345 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 133/264] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:09:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:08.284277 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b435da-1ac5-4c0f-94af-5bb194a2a18f None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:08.310280 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2270d4ff-b006-49e8-80ec-9d72a452e96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:08.311311 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270d4ff-b006-49e8-80ec-9d72a452e96c None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:08.311821 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270d4ff-b006-49e8-80ec-9d72a452e96c None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:08.312276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270d4ff-b006-49e8-80ec-9d72a452e96c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:08.313367 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2270d4ff-b006-49e8-80ec-9d72a452e96c None admin] RBAC: Adding query filter params (role.id=b4c524dd7bb44e94bd3b42d86e1cdf42, scope.project.id=5e327e3411214877a9f427330ef40dc5, user.id=8b1afed4cb0342a783c95efb73a8b869) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:08.313896 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2270d4ff-b006-49e8-80ec-9d72a452e96c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:08.315595 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2270d4ff-b006-49e8-80ec-9d72a452e96c None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:08.326207 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 132/265] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:09:08 2022] GET /identity/v3/role_assignments?user.id=8b1afed4cb0342a783c95efb73a8b869&scope.project.id=5e327e3411214877a9f427330ef40dc5&role.id=b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 608 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:08.371031 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:08.416022 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b435da-1ac5-4c0f-94af-5bb194a2a18f None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:08.455339 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e05c5-66cf-492f-a1e8-ef77a7fe3ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:08.456145 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e05c5-66cf-492f-a1e8-ef77a7fe3ada None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:08.456422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e05c5-66cf-492f-a1e8-ef77a7fe3ada None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:08.456690 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e05c5-66cf-492f-a1e8-ef77a7fe3ada None admin] PATH_INFO: `/v3/users/8b1afed4cb0342a783c95efb73a8b869` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:08.457939 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e05c5-66cf-492f-a1e8-ef77a7fe3ada None admin] RBAC: Authorizing `identity:get_user(user_id=8b1afed4cb0342a783c95efb73a8b869)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:08.459564 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e05c5-66cf-492f-a1e8-ef77a7fe3ada None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:08.461865 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 134/266] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:09:08 2022] GET /identity/v3/users/8b1afed4cb0342a783c95efb73a8b869 => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:08.467495 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2270d4ff-b006-49e8-80ec-9d72a452e96c None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:08.578436 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2270d4ff-b006-49e8-80ec-9d72a452e96c None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:08.581267 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:08.618360 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd88af20-5633-4c1e-8c3f-d8097afbea8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:08.619025 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88af20-5633-4c1e-8c3f-d8097afbea8d None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:08.619303 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88af20-5633-4c1e-8c3f-d8097afbea8d None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:08.619573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88af20-5633-4c1e-8c3f-d8097afbea8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:08.694043 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd88af20-5633-4c1e-8c3f-d8097afbea8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:08.695954 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd88af20-5633-4c1e-8c3f-d8097afbea8d None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:08.800695 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 133/267] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:09:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 03:09:08.806538 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e05c5-66cf-492f-a1e8-ef77a7fe3ada None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:08.912265 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e05c5-66cf-492f-a1e8-ef77a7fe3ada None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:08.942124 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:08.952896 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c21965-9aea-4f42-8115-b2e1ac688a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:08.954477 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c21965-9aea-4f42-8115-b2e1ac688a47 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:08.955336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c21965-9aea-4f42-8115-b2e1ac688a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:08.955995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c21965-9aea-4f42-8115-b2e1ac688a47 None admin] PATH_INFO: `/v3/projects/5e327e3411214877a9f427330ef40dc5` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:08.957941 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c21965-9aea-4f42-8115-b2e1ac688a47 None admin] RBAC: Authorizing `identity:get_project(project_id=5e327e3411214877a9f427330ef40dc5)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:08.959699 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c21965-9aea-4f42-8115-b2e1ac688a47 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:08.961833 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 135/268] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:09:08 2022] GET /identity/v3/projects/5e327e3411214877a9f427330ef40dc5 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:08.969110 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd88af20-5633-4c1e-8c3f-d8097afbea8d None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:08.971446 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:09.132289 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd88af20-5633-4c1e-8c3f-d8097afbea8d None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:09.175212 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b4d26-ff2f-4d7f-921b-f6a72e1a272c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:09.175887 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b4d26-ff2f-4d7f-921b-f6a72e1a272c None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:09.176171 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b4d26-ff2f-4d7f-921b-f6a72e1a272c None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:09.176450 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b4d26-ff2f-4d7f-921b-f6a72e1a272c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:09.177322 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534b4d26-ff2f-4d7f-921b-f6a72e1a272c None admin] RBAC: Adding query filter params (role.id=9e63b73b3ce24bb3ad214ecdb6132696, scope.project.id=5e327e3411214877a9f427330ef40dc5, user.id=8b1afed4cb0342a783c95efb73a8b869) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:09.177636 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534b4d26-ff2f-4d7f-921b-f6a72e1a272c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:09.179329 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534b4d26-ff2f-4d7f-921b-f6a72e1a272c None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:09.188818 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 134/269] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:09:08 2022] GET /identity/v3/role_assignments?user.id=8b1afed4cb0342a783c95efb73a8b869&scope.project.id=5e327e3411214877a9f427330ef40dc5&role.id=9e63b73b3ce24bb3ad214ecdb6132696 => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:09.289201 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544a95d-eda7-40ff-8d81-62e4924ed8fc None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:09.289758 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544a95d-eda7-40ff-8d81-62e4924ed8fc None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:09.289758 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544a95d-eda7-40ff-8d81-62e4924ed8fc None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:09.291474 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 136/270] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:09.297066 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f234573-3077-447c-bbb0-606d0bf50443 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:09.297356 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f234573-3077-447c-bbb0-606d0bf50443 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:09.297641 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f234573-3077-447c-bbb0-606d0bf50443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:09.325538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-7f234573-3077-447c-bbb0-606d0bf50443 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:09.327104 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-7f234573-3077-447c-bbb0-606d0bf50443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:09.422312 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:09.432310 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 135/271] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:09.445621 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d0e37-90d9-4521-9f7f-be350f148a94 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:09.446248 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d0e37-90d9-4521-9f7f-be350f148a94 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:09.446711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d0e37-90d9-4521-9f7f-be350f148a94 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:09.448323 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 137/272] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:09.454201 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba361ef-3a9c-4721-a552-eb9615ceb0a0 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:09.454741 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba361ef-3a9c-4721-a552-eb9615ceb0a0 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:09.455206 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba361ef-3a9c-4721-a552-eb9615ceb0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:09.465741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d0e37-90d9-4521-9f7f-be350f148a94 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:09.482208 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-6ba361ef-3a9c-4721-a552-eb9615ceb0a0 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:09.483809 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-6ba361ef-3a9c-4721-a552-eb9615ceb0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:09.569448 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 136/273] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:09.596050 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d0e37-90d9-4521-9f7f-be350f148a94 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:09.597216 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba361ef-3a9c-4721-a552-eb9615ceb0a0 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:09.642626 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c2bc2-4b80-4b8b-a1f7-b57e72f96a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:09.643437 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c2bc2-4b80-4b8b-a1f7-b57e72f96a6e None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:09.643715 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c2bc2-4b80-4b8b-a1f7-b57e72f96a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:09.643987 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c2bc2-4b80-4b8b-a1f7-b57e72f96a6e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:09.659226 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c2bc2-4b80-4b8b-a1f7-b57e72f96a6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:09.671125 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-d49c2bc2-4b80-4b8b-a1f7-b57e72f96a6e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:09:09.671125 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:09.671125 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:09.671125 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:09.671125 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:09.671125 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:09.671125 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:09.671125 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:09.671125 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:09.671125 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:09.671125 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:09.671125 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:09.671125 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:09.671125 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:09.671125 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:09.671125 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:09.672079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:09.672079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:09.672079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:09.672079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:09.672079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:09.672079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:09.672079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:09.672079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:09.672079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:09.672079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:09.672079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:09.672079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:09.672079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:09.672079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:09.672079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:09.672079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:09.672889 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:09.672889 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:09.672889 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:09.672889 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:09.672889 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:09.672889 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:09.672889 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:09.672889 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:09.672889 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:09.672889 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:09.672889 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:09:09.672889 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:09.672889 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 138/274] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:09:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:09.682284 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c2bc2-4b80-4b8b-a1f7-b57e72f96a6e None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:09.731222 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba361ef-3a9c-4721-a552-eb9615ceb0a0 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:09.755781 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:09.773297 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:09.784349 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f3958-a303-48d5-897a-05f1f21f59d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:09.786045 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f3958-a303-48d5-897a-05f1f21f59d1 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:09.786468 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f3958-a303-48d5-897a-05f1f21f59d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:09.787106 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f3958-a303-48d5-897a-05f1f21f59d1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:09.789390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2f3958-a303-48d5-897a-05f1f21f59d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:09.791584 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2f3958-a303-48d5-897a-05f1f21f59d1 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:09.793812 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 137/275] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:09.801891 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f3958-a303-48d5-897a-05f1f21f59d1 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:09.816760 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c2bc2-4b80-4b8b-a1f7-b57e72f96a6e None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:09.852329 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c25fea-743e-41fa-a309-35d27ac06e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:09.853186 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c25fea-743e-41fa-a309-35d27ac06e6e None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:09.853559 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c25fea-743e-41fa-a309-35d27ac06e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:09.853926 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c25fea-743e-41fa-a309-35d27ac06e6e None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:09.855090 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c25fea-743e-41fa-a309-35d27ac06e6e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:09.855208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c25fea-743e-41fa-a309-35d27ac06e6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:09.856700 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c25fea-743e-41fa-a309-35d27ac06e6e None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:09.866164 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 139/276] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 03:09:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:09.872142 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c25fea-743e-41fa-a309-35d27ac06e6e None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:09.921850 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f3958-a303-48d5-897a-05f1f21f59d1 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:09.960970 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa2f37-4602-46fd-9cd2-e8607cd70642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:09.961578 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa2f37-4602-46fd-9cd2-e8607cd70642 None admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:09.961860 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa2f37-4602-46fd-9cd2-e8607cd70642 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:09.962135 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa2f37-4602-46fd-9cd2-e8607cd70642 None admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:09.962875 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaa2f37-4602-46fd-9cd2-e8607cd70642 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:09.964313 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaa2f37-4602-46fd-9cd2-e8607cd70642 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:09.986788 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 138/277] 10.222.0.50 () {60 vars in 1132 bytes} [Fri May 27 03:09:09 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:09:09.993769 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c25fea-743e-41fa-a309-35d27ac06e6e None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:09.993864 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3309c-7741-4e9d-a5c8-ebd54c7eb8c5 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:09.993864 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3309c-7741-4e9d-a5c8-ebd54c7eb8c5 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:09.993864 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3309c-7741-4e9d-a5c8-ebd54c7eb8c5 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:09.993864 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 139/278] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:09.996085 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab109a5-1e7c-408b-8ca9-5f823a8419a2 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:09.997291 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab109a5-1e7c-408b-8ca9-5f823a8419a2 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:09.997291 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab109a5-1e7c-408b-8ca9-5f823a8419a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:10.018470 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-6ab109a5-1e7c-408b-8ca9-5f823a8419a2 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:10.019706 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-6ab109a5-1e7c-408b-8ca9-5f823a8419a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:10.029525 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:10.037789 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b278e4c4-3369-4985-a767-3ceb424a1236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:10.038842 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b278e4c4-3369-4985-a767-3ceb424a1236 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:10.039610 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b278e4c4-3369-4985-a767-3ceb424a1236 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:10.039905 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b278e4c4-3369-4985-a767-3ceb424a1236 None admin] PATH_INFO: `/v3/users/81e478bc272140e79c7ca72302862581` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:10.041744 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b278e4c4-3369-4985-a767-3ceb424a1236 None admin] RBAC: Authorizing `identity:get_user(user_id=81e478bc272140e79c7ca72302862581)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:10.044056 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b278e4c4-3369-4985-a767-3ceb424a1236 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:10.049254 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 140/279] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:09:09 2022] GET /identity/v3/users/81e478bc272140e79c7ca72302862581 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:10.053334 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b278e4c4-3369-4985-a767-3ceb424a1236 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:10.081954 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:10.113458 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 140/280] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:09:10.130406 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab109a5-1e7c-408b-8ca9-5f823a8419a2 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:10.172828 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b278e4c4-3369-4985-a767-3ceb424a1236 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:10.213517 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd053d97-bb74-4ebb-a650-d9ae68689e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:10.214859 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd053d97-bb74-4ebb-a650-d9ae68689e14 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:10.215350 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd053d97-bb74-4ebb-a650-d9ae68689e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:10.215829 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd053d97-bb74-4ebb-a650-d9ae68689e14 None admin] PATH_INFO: `/v3/projects/5e327e3411214877a9f427330ef40dc5` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:10.217183 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd053d97-bb74-4ebb-a650-d9ae68689e14 None admin] RBAC: Authorizing `identity:get_project(project_id=5e327e3411214877a9f427330ef40dc5)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:10.218759 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd053d97-bb74-4ebb-a650-d9ae68689e14 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:10.220588 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 141/281] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:09:10 2022] GET /identity/v3/projects/5e327e3411214877a9f427330ef40dc5 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:10.228387 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd053d97-bb74-4ebb-a650-d9ae68689e14 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:10.247708 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab109a5-1e7c-408b-8ca9-5f823a8419a2 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:10.308629 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a637eda-3803-44b1-8e6f-fdfd06049dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:10.309493 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a637eda-3803-44b1-8e6f-fdfd06049dd8 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:10.309772 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a637eda-3803-44b1-8e6f-fdfd06049dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:10.310045 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a637eda-3803-44b1-8e6f-fdfd06049dd8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:10.313621 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:10.330186 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a637eda-3803-44b1-8e6f-fdfd06049dd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:10.338547 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-5a637eda-3803-44b1-8e6f-fdfd06049dd8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:09:10.338547 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:10.338547 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:10.338547 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:10.338547 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:10.338547 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:10.338547 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:10.338547 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:10.338547 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:10.338547 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:10.338547 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:10.338547 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:10.338547 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:10.338547 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:10.338547 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:10.338547 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:10.339610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:10.339610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:10.339610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:10.339610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:10.339610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:10.339610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:10.339610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:10.339610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:10.339610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:10.339610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:10.339610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:10.339610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:10.339610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:10.339610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:10.339610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:10.339610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:10.340383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:10.340383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:10.340383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:10.340383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:10.340383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:10.340383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:10.340383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:10.340383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:10.340383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:10.340383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:10.340383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:09:10.340383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:10.345108 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 141/282] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:09:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:09:10.346147 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a637eda-3803-44b1-8e6f-fdfd06049dd8 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:10.377265 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd053d97-bb74-4ebb-a650-d9ae68689e14 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:10.397872 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:10.421630 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc96636-0f98-4e50-a15f-bbe79b9e984f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:10.422539 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc96636-0f98-4e50-a15f-bbe79b9e984f None admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:10.422821 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc96636-0f98-4e50-a15f-bbe79b9e984f None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:10.423091 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc96636-0f98-4e50-a15f-bbe79b9e984f None admin] PATH_INFO: `/v3/projects/5e327e3411214877a9f427330ef40dc5/users/81e478bc272140e79c7ca72302862581/roles/6479235705f74061aff23df5c9f89e33` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:10.426823 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc96636-0f98-4e50-a15f-bbe79b9e984f None admin] RBAC: Authorizing `identity:create_grant(project_id=5e327e3411214877a9f427330ef40dc5, user_id=81e478bc272140e79c7ca72302862581, role_id=6479235705f74061aff23df5c9f89e33)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:10.428507 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc96636-0f98-4e50-a15f-bbe79b9e984f None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:10.451992 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 142/283] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 03:09:10 2022] PUT /identity/v3/projects/5e327e3411214877a9f427330ef40dc5/users/81e478bc272140e79c7ca72302862581/roles/6479235705f74061aff23df5c9f89e33 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:09:10.486089 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a637eda-3803-44b1-8e6f-fdfd06049dd8 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:10.515978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1bffc9-8c6d-4f28-a3ac-2cb65c8f698c None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:10.516313 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1bffc9-8c6d-4f28-a3ac-2cb65c8f698c None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:10.516560 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1bffc9-8c6d-4f28-a3ac-2cb65c8f698c None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:10.518056 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 143/284] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:09:10.524582 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b1ad8-0f31-40f8-b06f-733b6449983e None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:10.524883 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b1ad8-0f31-40f8-b06f-733b6449983e None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:10.525166 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b1ad8-0f31-40f8-b06f-733b6449983e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:10.528788 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c2bb82-0cb7-4638-a175-53205e589a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:10.529543 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c2bb82-0cb7-4638-a175-53205e589a0d None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:10.529882 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c2bb82-0cb7-4638-a175-53205e589a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:10.530146 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c2bb82-0cb7-4638-a175-53205e589a0d None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:10.530942 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c2bb82-0cb7-4638-a175-53205e589a0d None admin] RBAC: Adding query filter params (name=member) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:10.531249 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c2bb82-0cb7-4638-a175-53205e589a0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:10.533194 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c2bb82-0cb7-4638-a175-53205e589a0d None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:10.547496 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-577b1ad8-0f31-40f8-b06f-733b6449983e None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:10.549080 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-577b1ad8-0f31-40f8-b06f-733b6449983e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:10.551502 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 142/285] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 03:09:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:10.554439 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c2bb82-0cb7-4638-a175-53205e589a0d None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:10.643669 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 144/286] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:09:10.684587 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c2bb82-0cb7-4638-a175-53205e589a0d None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:10.686690 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:10.688637 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577b1ad8-0f31-40f8-b06f-733b6449983e None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:10.738624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b219c-8dfa-4172-be94-3fefcad774e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:10.739478 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b219c-8dfa-4172-be94-3fefcad774e6 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:10.739764 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b219c-8dfa-4172-be94-3fefcad774e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:10.740037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042b219c-8dfa-4172-be94-3fefcad774e6 None admin] PATH_INFO: `/v3/users/156f94969c9a47b997490cb92ceb4953` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:10.765766 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:10.767868 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b219c-8dfa-4172-be94-3fefcad774e6 None admin] RBAC: Authorizing `identity:get_user(user_id=156f94969c9a47b997490cb92ceb4953)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:10.770681 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042b219c-8dfa-4172-be94-3fefcad774e6 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:10.774687 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 143/287] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:09:10 2022] GET /identity/v3/users/156f94969c9a47b997490cb92ceb4953 => generated 293 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:10.784131 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b219c-8dfa-4172-be94-3fefcad774e6 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:10.860321 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577b1ad8-0f31-40f8-b06f-733b6449983e None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:10.919512 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950903d0-a9ad-488c-9496-17aeb13281fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:10.920327 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950903d0-a9ad-488c-9496-17aeb13281fb None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:10.920602 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950903d0-a9ad-488c-9496-17aeb13281fb None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:10.920874 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950903d0-a9ad-488c-9496-17aeb13281fb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:10.948088 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950903d0-a9ad-488c-9496-17aeb13281fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:10.956856 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-950903d0-a9ad-488c-9496-17aeb13281fb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:09:10.956856 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:10.956856 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:10.956856 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:10.956856 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:10.956856 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:10.956856 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:10.956856 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:10.956856 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:10.956856 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:10.956856 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:10.956856 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:10.956856 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:10.956856 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:10.956856 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:10.956856 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:10.958460 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:10.958460 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:10.958460 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:10.958460 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:10.958460 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:10.958460 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:10.958460 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:10.958460 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:10.958460 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:10.958460 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:10.958460 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:10.958460 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:10.958460 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:10.958460 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:10.958460 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:10.958460 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:10.959963 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:10.959963 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:10.959963 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:10.959963 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:10.959963 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:10.959963 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:10.959963 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:10.959963 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:10.959963 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:10.959963 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:10.959963 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:09:10.959963 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:10.959963 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 145/288] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:09:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:10.971681 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950903d0-a9ad-488c-9496-17aeb13281fb None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:11.007422 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042b219c-8dfa-4172-be94-3fefcad774e6 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:11.077767 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dffbd6-7079-4289-bb64-4356c3f06814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:11.078464 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dffbd6-7079-4289-bb64-4356c3f06814 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:11.078745 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dffbd6-7079-4289-bb64-4356c3f06814 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:11.079021 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dffbd6-7079-4289-bb64-4356c3f06814 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:11.079873 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dffbd6-7079-4289-bb64-4356c3f06814 None admin] RBAC: Adding query filter params (role.id=0129dd45ff5449f9957e8efdd01a55ed, user.id=156f94969c9a47b997490cb92ceb4953, scope.system=all) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:11.080180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dffbd6-7079-4289-bb64-4356c3f06814 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:11.081746 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dffbd6-7079-4289-bb64-4356c3f06814 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:11.092235 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 144/289] 10.222.0.50 () {56 vars in 1308 bytes} [Fri May 27 03:09:10 2022] GET /identity/v3/role_assignments?user.id=156f94969c9a47b997490cb92ceb4953&scope.system=all&role.id=0129dd45ff5449f9957e8efdd01a55ed => generated 226 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:11.150349 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950903d0-a9ad-488c-9496-17aeb13281fb None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:11.160739 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:11.210041 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8cc008-8694-47e5-9a9d-23e9626661e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:11.211221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8cc008-8694-47e5-9a9d-23e9626661e9 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:11.211738 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8cc008-8694-47e5-9a9d-23e9626661e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:11.212219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8cc008-8694-47e5-9a9d-23e9626661e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:11.213245 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8cc008-8694-47e5-9a9d-23e9626661e9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:11.213756 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8cc008-8694-47e5-9a9d-23e9626661e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:11.215604 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8cc008-8694-47e5-9a9d-23e9626661e9 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:11.233848 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 146/290] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 03:09:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:11.239976 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dffbd6-7079-4289-bb64-4356c3f06814 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:11.313373 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:11.430103 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dffbd6-7079-4289-bb64-4356c3f06814 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:11.484568 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bf4a43-30bd-4947-a42c-61aa7de179a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:11.486691 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf4a43-30bd-4947-a42c-61aa7de179a1 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:11.487496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf4a43-30bd-4947-a42c-61aa7de179a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:11.488121 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf4a43-30bd-4947-a42c-61aa7de179a1 None admin] PATH_INFO: `/v3/users/10114c80d86f43bdb74ff9a5ffebcdd9` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:11.493749 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bf4a43-30bd-4947-a42c-61aa7de179a1 None admin] RBAC: Authorizing `identity:get_user(user_id=10114c80d86f43bdb74ff9a5ffebcdd9)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:11.496691 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bf4a43-30bd-4947-a42c-61aa7de179a1 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:11.499937 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 145/291] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:09:11 2022] GET /identity/v3/users/10114c80d86f43bdb74ff9a5ffebcdd9 => generated 297 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:11.505687 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8cc008-8694-47e5-9a9d-23e9626661e9 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:11.672985 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8cc008-8694-47e5-9a9d-23e9626661e9 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:11.724483 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d930b-5da2-46f7-ad25-71092515b84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:11.725198 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d930b-5da2-46f7-ad25-71092515b84a None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:11.725479 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d930b-5da2-46f7-ad25-71092515b84a None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:11.725753 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d930b-5da2-46f7-ad25-71092515b84a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:11.777183 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:11.832340 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d930b-5da2-46f7-ad25-71092515b84a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:11.833849 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d930b-5da2-46f7-ad25-71092515b84a None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:11.992413 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 147/292] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:09:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:09:11.998112 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bf4a43-30bd-4947-a42c-61aa7de179a1 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:12.103583 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:12.112150 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bf4a43-30bd-4947-a42c-61aa7de179a1 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:12.147541 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ba1777-bd76-47de-8f7c-ef081f75e695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:12.148431 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba1777-bd76-47de-8f7c-ef081f75e695 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:12.148712 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba1777-bd76-47de-8f7c-ef081f75e695 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:12.149007 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ba1777-bd76-47de-8f7c-ef081f75e695 None admin] PATH_INFO: `/v3/projects/5e327e3411214877a9f427330ef40dc5` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:12.150233 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ba1777-bd76-47de-8f7c-ef081f75e695 None admin] RBAC: Authorizing `identity:get_project(project_id=5e327e3411214877a9f427330ef40dc5)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:12.151703 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ba1777-bd76-47de-8f7c-ef081f75e695 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:12.153793 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 146/293] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:09:11 2022] GET /identity/v3/projects/5e327e3411214877a9f427330ef40dc5 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:12.161824 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d930b-5da2-46f7-ad25-71092515b84a None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:12.252693 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:12.311695 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d930b-5da2-46f7-ad25-71092515b84a None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:12.360425 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749686d9-a2eb-44ca-abe1-69459a277165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:12.361498 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749686d9-a2eb-44ca-abe1-69459a277165 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:12.361966 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749686d9-a2eb-44ca-abe1-69459a277165 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:12.362437 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749686d9-a2eb-44ca-abe1-69459a277165 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:12.363451 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749686d9-a2eb-44ca-abe1-69459a277165 None admin] RBAC: Adding query filter params (role.id=0129dd45ff5449f9957e8efdd01a55ed, scope.project.id=5e327e3411214877a9f427330ef40dc5, user.id=10114c80d86f43bdb74ff9a5ffebcdd9) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:12.363959 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749686d9-a2eb-44ca-abe1-69459a277165 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:12.365747 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749686d9-a2eb-44ca-abe1-69459a277165 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:12.372397 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 148/294] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:09:12 2022] GET /identity/v3/role_assignments?user.id=10114c80d86f43bdb74ff9a5ffebcdd9&scope.project.id=5e327e3411214877a9f427330ef40dc5&role.id=0129dd45ff5449f9957e8efdd01a55ed => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:13.038331 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306794e7-4cc0-4115-9ef6-d202fcc17f4d None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:13.038610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306794e7-4cc0-4115-9ef6-d202fcc17f4d None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:13.039158 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306794e7-4cc0-4115-9ef6-d202fcc17f4d None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:13.041125 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 147/295] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:13.046623 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916ff2e-8207-49d1-920a-43324cde7805 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:13.047487 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916ff2e-8207-49d1-920a-43324cde7805 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:13.047992 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916ff2e-8207-49d1-920a-43324cde7805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:13.070181 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-b916ff2e-8207-49d1-920a-43324cde7805 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:13.071742 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-b916ff2e-8207-49d1-920a-43324cde7805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:13.145163 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 149/296] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:13.169662 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306794e7-4cc0-4115-9ef6-d202fcc17f4d None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:13.180740 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8a07f-44ba-47f2-a830-1cb86268c193 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:13.181058 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8a07f-44ba-47f2-a830-1cb86268c193 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:13.181339 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8a07f-44ba-47f2-a830-1cb86268c193 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:13.183146 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 150/297] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:09:13.189663 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25889050-a16c-4d17-8d13-b05535328028 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:13.189950 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25889050-a16c-4d17-8d13-b05535328028 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:13.190230 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25889050-a16c-4d17-8d13-b05535328028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:13.210575 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-25889050-a16c-4d17-8d13-b05535328028 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:13.211759 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-25889050-a16c-4d17-8d13-b05535328028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:13.273877 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:13.284380 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 151/298] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:09:13.286334 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36a9d0-6745-46fa-88d3-9c525bafd911 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:13.286745 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36a9d0-6745-46fa-88d3-9c525bafd911 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:13.287204 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36a9d0-6745-46fa-88d3-9c525bafd911 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:13.288691 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 152/299] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:13.294196 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e94492-a0e2-40d1-bc6c-501b662839f3 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:13.294852 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e94492-a0e2-40d1-bc6c-501b662839f3 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:13.295266 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e94492-a0e2-40d1-bc6c-501b662839f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:13.297359 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306794e7-4cc0-4115-9ef6-d202fcc17f4d None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:13.320607 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-75e94492-a0e2-40d1-bc6c-501b662839f3 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:13.321813 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-75e94492-a0e2-40d1-bc6c-501b662839f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:13.336303 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e415a6-78d6-472e-a012-e56263a722dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:13.337162 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e415a6-78d6-472e-a012-e56263a722dc None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:13.337445 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e415a6-78d6-472e-a012-e56263a722dc None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:13.337718 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e415a6-78d6-472e-a012-e56263a722dc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:13.338899 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e415a6-78d6-472e-a012-e56263a722dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:13.340478 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e415a6-78d6-472e-a012-e56263a722dc None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:13.342347 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 148/300] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:13.347572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e415a6-78d6-472e-a012-e56263a722dc None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:13.377691 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:13.392844 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 153/301] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:09:13.397973 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e94492-a0e2-40d1-bc6c-501b662839f3 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:13.459265 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e415a6-78d6-472e-a012-e56263a722dc None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:13.493325 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11af5a0-c6fc-4f18-896c-cb854a2d1c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:13.494475 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11af5a0-c6fc-4f18-896c-cb854a2d1c51 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:13.494988 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11af5a0-c6fc-4f18-896c-cb854a2d1c51 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:13.495324 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11af5a0-c6fc-4f18-896c-cb854a2d1c51 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:13.507079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e94492-a0e2-40d1-bc6c-501b662839f3 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:13.507486 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11af5a0-c6fc-4f18-896c-cb854a2d1c51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:13.515732 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-b11af5a0-c6fc-4f18-896c-cb854a2d1c51 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:09:13.515732 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:13.515732 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:13.515732 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:13.515732 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:13.515732 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:13.515732 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:13.515732 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:13.515732 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:13.515732 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:13.515732 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:13.515732 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:13.515732 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:13.515732 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:13.515732 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:13.515732 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:13.516590 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:13.516590 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:13.516590 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:13.516590 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:13.516590 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:13.516590 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:13.516590 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:13.516590 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:13.516590 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:13.516590 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:13.516590 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:13.516590 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:13.516590 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:13.516590 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:13.516590 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:13.516590 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:13.517429 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:13.517429 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:13.517429 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:13.517429 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:13.517429 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:13.517429 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:13.517429 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:13.517429 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:13.517429 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:13.517429 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:13.517429 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:09:13.517429 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:13.517429 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 149/302] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:09:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:13.518604 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11af5a0-c6fc-4f18-896c-cb854a2d1c51 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:13.533144 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:13.546236 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f650d6f3-b31e-4ae2-a513-bc3be777903a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:13.546855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650d6f3-b31e-4ae2-a513-bc3be777903a None admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:13.547140 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650d6f3-b31e-4ae2-a513-bc3be777903a None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:13.547415 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650d6f3-b31e-4ae2-a513-bc3be777903a None admin] PATH_INFO: `/v3/groups` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:13.548097 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f650d6f3-b31e-4ae2-a513-bc3be777903a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:13.549597 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f650d6f3-b31e-4ae2-a513-bc3be777903a None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:13.567348 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 154/303] 10.222.0.50 () {60 vars in 1134 bytes} [Fri May 27 03:09:13 2022] POST /identity/v3/groups => generated 229 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:09:13.568655 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f650d6f3-b31e-4ae2-a513-bc3be777903a None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:13.639820 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:13.658039 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11af5a0-c6fc-4f18-896c-cb854a2d1c51 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:13.698629 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89c44f-b0b5-4636-a65d-43df5b9aa1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:13.699449 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89c44f-b0b5-4636-a65d-43df5b9aa1ac None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:13.699730 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89c44f-b0b5-4636-a65d-43df5b9aa1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:13.700004 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89c44f-b0b5-4636-a65d-43df5b9aa1ac None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:13.712833 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e89c44f-b0b5-4636-a65d-43df5b9aa1ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:13.720853 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-6e89c44f-b0b5-4636-a65d-43df5b9aa1ac None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:09:13.720853 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:13.720853 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:13.720853 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:13.720853 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:13.720853 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:13.720853 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:13.720853 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:13.720853 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:13.720853 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:13.720853 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:13.720853 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:13.720853 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:13.720853 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:13.720853 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:13.720853 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:13.721834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:13.721834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:13.721834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:13.721834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:13.721834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:13.721834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:13.721834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:13.721834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:13.721834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:13.721834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:13.721834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:13.721834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:13.721834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:13.721834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:13.721834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:13.721834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:13.722610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:13.722610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:13.722610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:13.722610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:13.722610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:13.722610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:13.722610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:13.722610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:13.722610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:13.722610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:13.722610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:09:13.722610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:13.722610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 150/304] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:09:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:13.724909 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f650d6f3-b31e-4ae2-a513-bc3be777903a None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:13.728448 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89c44f-b0b5-4636-a65d-43df5b9aa1ac None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:13.763053 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733270bc-481e-4590-9244-699841f98636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:13.763762 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733270bc-481e-4590-9244-699841f98636 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:13.764029 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733270bc-481e-4590-9244-699841f98636 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:13.764305 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733270bc-481e-4590-9244-699841f98636 None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:13.765132 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733270bc-481e-4590-9244-699841f98636 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:13.765750 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733270bc-481e-4590-9244-699841f98636 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:13.766903 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733270bc-481e-4590-9244-699841f98636 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:13.776775 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 155/305] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 03:09:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:13.782526 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733270bc-481e-4590-9244-699841f98636 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:13.829513 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:13.851030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89c44f-b0b5-4636-a65d-43df5b9aa1ac None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:13.885095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41349469-e144-40f5-b381-0c1de54017ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:13.885774 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41349469-e144-40f5-b381-0c1de54017ef None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:13.886055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41349469-e144-40f5-b381-0c1de54017ef None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:13.886333 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41349469-e144-40f5-b381-0c1de54017ef None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:13.887085 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41349469-e144-40f5-b381-0c1de54017ef None admin] RBAC: Adding query filter params (name=reader) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:13.887389 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41349469-e144-40f5-b381-0c1de54017ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:13.888793 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41349469-e144-40f5-b381-0c1de54017ef None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:13.892840 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733270bc-481e-4590-9244-699841f98636 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:13.901610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 151/306] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 03:09:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:13.901610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a6ae8-fc1d-41b1-ac9d-dfc1c3031ea2 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:13.901610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a6ae8-fc1d-41b1-ac9d-dfc1c3031ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:13.901610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a6ae8-fc1d-41b1-ac9d-dfc1c3031ea2 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:13.901610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 152/307] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:13.905286 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a6ae8-fc1d-41b1-ac9d-dfc1c3031ea2 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:13.918796 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:13.940243 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d08aac-c0f6-4ded-b137-e170cc9785a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:13.941473 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d08aac-c0f6-4ded-b137-e170cc9785a2 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:13.941895 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d08aac-c0f6-4ded-b137-e170cc9785a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:13.942175 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d08aac-c0f6-4ded-b137-e170cc9785a2 None admin] PATH_INFO: `/v3/users/81e478bc272140e79c7ca72302862581` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:13.943365 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d08aac-c0f6-4ded-b137-e170cc9785a2 None admin] RBAC: Authorizing `identity:get_user(user_id=81e478bc272140e79c7ca72302862581)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:13.945040 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d08aac-c0f6-4ded-b137-e170cc9785a2 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:13.951460 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 156/308] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:09:13 2022] GET /identity/v3/users/81e478bc272140e79c7ca72302862581 => generated 297 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:13.951460 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8871019-1eb3-47db-9fd0-756ba249fd7f None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:13.951460 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8871019-1eb3-47db-9fd0-756ba249fd7f None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:13.951460 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8871019-1eb3-47db-9fd0-756ba249fd7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:13.968934 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-b8871019-1eb3-47db-9fd0-756ba249fd7f None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:13.970223 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-b8871019-1eb3-47db-9fd0-756ba249fd7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:14.022769 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a6ae8-fc1d-41b1-ac9d-dfc1c3031ea2 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:14.045770 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 157/309] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:14.050150 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8871019-1eb3-47db-9fd0-756ba249fd7f None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:14.060625 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbffc08-6b3b-4701-9133-9696b358291a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:14.061545 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbffc08-6b3b-4701-9133-9696b358291a None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:14.061825 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbffc08-6b3b-4701-9133-9696b358291a None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:14.062099 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbffc08-6b3b-4701-9133-9696b358291a None admin] PATH_INFO: `/v3/users/2d249e51ac5f4a4fb3cdc21010bd3d7c` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:14.076943 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbffc08-6b3b-4701-9133-9696b358291a None admin] RBAC: Authorizing `identity:get_user(user_id=2d249e51ac5f4a4fb3cdc21010bd3d7c)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:14.078458 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbffc08-6b3b-4701-9133-9696b358291a None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:14.080254 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 153/310] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:09:13 2022] GET /identity/v3/users/2d249e51ac5f4a4fb3cdc21010bd3d7c => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:14.081116 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbffc08-6b3b-4701-9133-9696b358291a None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:14.109587 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:14.154479 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:14.173796 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8871019-1eb3-47db-9fd0-756ba249fd7f None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:14.190808 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbffc08-6b3b-4701-9133-9696b358291a None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:14.208937 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d3f93a-bd33-49b3-928b-761c51b516e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:14.209645 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3f93a-bd33-49b3-928b-761c51b516e1 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:14.209923 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3f93a-bd33-49b3-928b-761c51b516e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:14.210199 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d3f93a-bd33-49b3-928b-761c51b516e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:14.227782 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622044f-5ac7-4aa5-8871-adefdc1aa0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:14.228682 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622044f-5ac7-4aa5-8871-adefdc1aa0ff None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:14.228879 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622044f-5ac7-4aa5-8871-adefdc1aa0ff None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:14.229184 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622044f-5ac7-4aa5-8871-adefdc1aa0ff None admin] PATH_INFO: `/v3/roles/member` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:14.240347 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f622044f-5ac7-4aa5-8871-adefdc1aa0ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:14.248278 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-f622044f-5ac7-4aa5-8871-adefdc1aa0ff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:09:14.248278 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:14.248278 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:14.248278 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:14.248278 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:14.248278 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:14.248278 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:14.248278 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:14.248278 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:14.248278 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:14.248278 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:14.248278 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:14.248278 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:14.248278 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:14.248278 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:14.248278 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:14.249401 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:14.249401 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:14.249401 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:14.249401 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:14.249401 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:14.249401 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:14.249401 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:14.249401 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:14.249401 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:14.249401 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:14.249401 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:14.249401 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:14.249401 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:14.249401 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:14.249401 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:14.249401 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:14.250199 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:14.250199 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:14.250199 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:14.250199 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:14.250199 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:14.250199 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:14.250199 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:14.250199 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:14.250199 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:14.250199 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:14.250199 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:09:14.250199 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:14.250199 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 154/311] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:09:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:14.250830 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622044f-5ac7-4aa5-8871-adefdc1aa0ff None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:14.287947 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d3f93a-bd33-49b3-928b-761c51b516e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:14.290005 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d3f93a-bd33-49b3-928b-761c51b516e1 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:14.340676 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:14.364308 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622044f-5ac7-4aa5-8871-adefdc1aa0ff None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:14.395230 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:14.405593 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ca56e-eb1b-40e9-af21-7a987fd28a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:14.406224 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ca56e-eb1b-40e9-af21-7a987fd28a51 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:14.406592 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ca56e-eb1b-40e9-af21-7a987fd28a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:14.406763 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ca56e-eb1b-40e9-af21-7a987fd28a51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:14.407600 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ca56e-eb1b-40e9-af21-7a987fd28a51 None admin] RBAC: Adding query filter params (role.id=6479235705f74061aff23df5c9f89e33, user.id=2d249e51ac5f4a4fb3cdc21010bd3d7c, scope.system=all) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:14.407909 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ca56e-eb1b-40e9-af21-7a987fd28a51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:14.409578 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ca56e-eb1b-40e9-af21-7a987fd28a51 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:14.412841 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 158/312] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:09:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:09:14.413732 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d3f93a-bd33-49b3-928b-761c51b516e1 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:14.429002 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 155/313] 10.222.0.50 () {56 vars in 1308 bytes} [Fri May 27 03:09:14 2022] GET /identity/v3/role_assignments?user.id=2d249e51ac5f4a4fb3cdc21010bd3d7c&scope.system=all&role.id=6479235705f74061aff23df5c9f89e33 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:14.429002 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ca56e-eb1b-40e9-af21-7a987fd28a51 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:14.529071 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d3f93a-bd33-49b3-928b-761c51b516e1 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:14.567639 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f9ea4c-4447-4fb2-8bba-0c177d601139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:14.568324 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9ea4c-4447-4fb2-8bba-0c177d601139 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:14.568780 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9ea4c-4447-4fb2-8bba-0c177d601139 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:14.569124 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9ea4c-4447-4fb2-8bba-0c177d601139 None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:14.569889 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f9ea4c-4447-4fb2-8bba-0c177d601139 None admin] RBAC: Adding query filter params (name=member) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:14.570199 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f9ea4c-4447-4fb2-8bba-0c177d601139 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:14.571633 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f9ea4c-4447-4fb2-8bba-0c177d601139 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:14.578265 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ca56e-eb1b-40e9-af21-7a987fd28a51 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:14.581305 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 159/314] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 03:09:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:14.587180 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f9ea4c-4447-4fb2-8bba-0c177d601139 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:14.592279 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:14.627879 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dd5ae4-2fe1-40a6-93b1-49e14805f88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:14.629213 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd5ae4-2fe1-40a6-93b1-49e14805f88f None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:14.629494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd5ae4-2fe1-40a6-93b1-49e14805f88f None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:14.630772 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd5ae4-2fe1-40a6-93b1-49e14805f88f None admin] PATH_INFO: `/v3/projects/5e327e3411214877a9f427330ef40dc5` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:14.632237 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dd5ae4-2fe1-40a6-93b1-49e14805f88f None admin] RBAC: Authorizing `identity:get_project(project_id=5e327e3411214877a9f427330ef40dc5)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:14.634299 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dd5ae4-2fe1-40a6-93b1-49e14805f88f None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:14.640230 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 156/315] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:09:14 2022] GET /identity/v3/projects/5e327e3411214877a9f427330ef40dc5 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:14.641367 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dd5ae4-2fe1-40a6-93b1-49e14805f88f None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:14.698409 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f9ea4c-4447-4fb2-8bba-0c177d601139 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:14.719168 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:14.739076 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564ed7c-bcfa-44c1-a4ab-e47c40e4f140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:14.740333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564ed7c-bcfa-44c1-a4ab-e47c40e4f140 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:14.740834 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564ed7c-bcfa-44c1-a4ab-e47c40e4f140 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:14.741358 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564ed7c-bcfa-44c1-a4ab-e47c40e4f140 None admin] PATH_INFO: `/v3/users/156f94969c9a47b997490cb92ceb4953` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:14.742873 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3564ed7c-bcfa-44c1-a4ab-e47c40e4f140 None admin] RBAC: Authorizing `identity:get_user(user_id=156f94969c9a47b997490cb92ceb4953)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:14.744571 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3564ed7c-bcfa-44c1-a4ab-e47c40e4f140 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:14.746461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 160/316] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:09:14 2022] GET /identity/v3/users/156f94969c9a47b997490cb92ceb4953 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:14.751886 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564ed7c-bcfa-44c1-a4ab-e47c40e4f140 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:14.778342 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dd5ae4-2fe1-40a6-93b1-49e14805f88f None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:14.817389 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f90b48-b8fd-4948-b469-101af3f73819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:14.818566 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f90b48-b8fd-4948-b469-101af3f73819 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:14.819245 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f90b48-b8fd-4948-b469-101af3f73819 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:14.819850 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f90b48-b8fd-4948-b469-101af3f73819 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:14.820886 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f90b48-b8fd-4948-b469-101af3f73819 None admin] RBAC: Adding query filter params (role.id=6479235705f74061aff23df5c9f89e33, scope.project.id=5e327e3411214877a9f427330ef40dc5, user.id=81e478bc272140e79c7ca72302862581) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:14.821407 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f90b48-b8fd-4948-b469-101af3f73819 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:14.823007 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f90b48-b8fd-4948-b469-101af3f73819 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:14.825889 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:14.830064 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 157/317] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:09:14 2022] GET /identity/v3/role_assignments?user.id=81e478bc272140e79c7ca72302862581&scope.project.id=5e327e3411214877a9f427330ef40dc5&role.id=6479235705f74061aff23df5c9f89e33 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:14.864987 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564ed7c-bcfa-44c1-a4ab-e47c40e4f140 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:14.904443 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6cc4c-8035-4613-beaf-17ede3976e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:14.905632 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6cc4c-8035-4613-beaf-17ede3976e98 None admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:14.906051 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6cc4c-8035-4613-beaf-17ede3976e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:14.906490 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6cc4c-8035-4613-beaf-17ede3976e98 None admin] PATH_INFO: `/v3/system/users/156f94969c9a47b997490cb92ceb4953/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:14.908244 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f6cc4c-8035-4613-beaf-17ede3976e98 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=156f94969c9a47b997490cb92ceb4953, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:14.909883 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f6cc4c-8035-4613-beaf-17ede3976e98 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:14.923535 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 161/318] 10.222.0.50 () {58 vars in 1334 bytes} [Fri May 27 03:09:14 2022] PUT /identity/v3/system/users/156f94969c9a47b997490cb92ceb4953/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:09:15.833175 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f17d0-b074-4c25-81f7-9be18bf81a65 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:15.833751 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f17d0-b074-4c25-81f7-9be18bf81a65 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:15.833751 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f17d0-b074-4c25-81f7-9be18bf81a65 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:15.835487 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 158/319] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:15.840650 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a97c8-60dc-44d3-943c-b48970459679 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:15.841091 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a97c8-60dc-44d3-943c-b48970459679 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:15.841273 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a97c8-60dc-44d3-943c-b48970459679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:15.859108 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-ac3a97c8-60dc-44d3-943c-b48970459679 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:15.860282 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-ac3a97c8-60dc-44d3-943c-b48970459679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:15.929478 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 162/320] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:15.951441 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f17d0-b074-4c25-81f7-9be18bf81a65 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:16.057239 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f17d0-b074-4c25-81f7-9be18bf81a65 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:16.080863 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:16.093322 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4aadd8c-d21b-42f2-af3d-f172bd0c0f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:16.094062 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aadd8c-d21b-42f2-af3d-f172bd0c0f90 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:16.094348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aadd8c-d21b-42f2-af3d-f172bd0c0f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:16.094620 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4aadd8c-d21b-42f2-af3d-f172bd0c0f90 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:16.095772 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aadd8c-d21b-42f2-af3d-f172bd0c0f90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:16.097246 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4aadd8c-d21b-42f2-af3d-f172bd0c0f90 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:16.099073 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 159/321] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:16.105158 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3a97c8-60dc-44d3-943c-b48970459679 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:16.175297 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:16.213730 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3a97c8-60dc-44d3-943c-b48970459679 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:16.249092 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e466a851-e741-4849-930f-c803a88a4712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:16.249760 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466a851-e741-4849-930f-c803a88a4712 None admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:16.250036 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466a851-e741-4849-930f-c803a88a4712 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:16.250307 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466a851-e741-4849-930f-c803a88a4712 None admin] PATH_INFO: `/v3/groups` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:16.251001 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e466a851-e741-4849-930f-c803a88a4712 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:16.252449 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e466a851-e741-4849-930f-c803a88a4712 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:16.268934 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 163/322] 10.222.0.50 () {60 vars in 1134 bytes} [Fri May 27 03:09:16 2022] POST /identity/v3/groups => generated 226 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:09:16.667664 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b246b4-7a86-4aa8-8736-9db33e5652b6 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:16.667971 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b246b4-7a86-4aa8-8736-9db33e5652b6 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:16.668243 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b246b4-7a86-4aa8-8736-9db33e5652b6 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:16.669884 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 160/323] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:16.675531 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b80a1-8349-4ca9-b0de-ea1c2fb7afd6 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:16.676102 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b80a1-8349-4ca9-b0de-ea1c2fb7afd6 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:16.676638 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b80a1-8349-4ca9-b0de-ea1c2fb7afd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:16.695183 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-d82b80a1-8349-4ca9-b0de-ea1c2fb7afd6 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:16.696760 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-d82b80a1-8349-4ca9-b0de-ea1c2fb7afd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:16.785897 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 164/324] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:16.809356 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b246b4-7a86-4aa8-8736-9db33e5652b6 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:16.921850 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b246b4-7a86-4aa8-8736-9db33e5652b6 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:16.963984 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80930931-c31c-4936-b657-e5c6c64ef8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:16.965308 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80930931-c31c-4936-b657-e5c6c64ef8b0 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:16.966864 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80930931-c31c-4936-b657-e5c6c64ef8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:16.967392 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80930931-c31c-4936-b657-e5c6c64ef8b0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:16.983102 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80930931-c31c-4936-b657-e5c6c64ef8b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:16.991870 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-80930931-c31c-4936-b657-e5c6c64ef8b0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:09:16.991870 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:16.991870 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:16.991870 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:16.991870 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:16.991870 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:16.991870 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:16.991870 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:16.991870 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:16.991870 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:16.991870 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:16.991870 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:16.991870 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:16.991870 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:16.991870 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:16.991870 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:16.992884 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:16.992884 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:16.992884 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:16.992884 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:16.992884 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:16.992884 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:16.992884 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:16.992884 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:16.992884 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:16.992884 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:16.992884 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:16.992884 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:16.992884 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:16.992884 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:16.992884 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:16.992884 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:16.993758 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:16.993758 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:16.993758 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:16.993758 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:16.993758 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:16.993758 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:16.993758 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:16.993758 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:16.993758 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:16.993758 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:16.993758 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:09:16.993758 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:16.996268 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 161/325] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:09:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:17.005492 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82b80a1-8349-4ca9-b0de-ea1c2fb7afd6 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:17.067863 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:17.126486 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82b80a1-8349-4ca9-b0de-ea1c2fb7afd6 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:17.167794 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5280b4a-853a-419e-bfa2-c2c44b711611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:17.168475 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5280b4a-853a-419e-bfa2-c2c44b711611 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:17.168749 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5280b4a-853a-419e-bfa2-c2c44b711611 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:17.169053 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5280b4a-853a-419e-bfa2-c2c44b711611 None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:17.169799 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5280b4a-853a-419e-bfa2-c2c44b711611 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:17.171114 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5280b4a-853a-419e-bfa2-c2c44b711611 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:17.171546 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6348ad-4cd4-4a04-94d9-0693171d4605 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:17.171824 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6348ad-4cd4-4a04-94d9-0693171d4605 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:17.172145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6348ad-4cd4-4a04-94d9-0693171d4605 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:17.173512 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5280b4a-853a-419e-bfa2-c2c44b711611 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:17.173732 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 162/326] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:17.179933 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980bf864-3a88-4438-b0b7-df7ef32f0ed1 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:17.180221 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980bf864-3a88-4438-b0b7-df7ef32f0ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:17.180496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980bf864-3a88-4438-b0b7-df7ef32f0ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:17.186691 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 165/327] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 03:09:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:17.192406 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5280b4a-853a-419e-bfa2-c2c44b711611 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:17.200378 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-980bf864-3a88-4438-b0b7-df7ef32f0ed1 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:17.201611 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-980bf864-3a88-4438-b0b7-df7ef32f0ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:17.262562 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:17.319851 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5280b4a-853a-419e-bfa2-c2c44b711611 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:17.326388 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 163/328] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:17.338690 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:17.355895 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2ab2f6-3da7-4489-a984-8edb564160ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:17.356790 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ab2f6-3da7-4489-a984-8edb564160ee None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:17.357099 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ab2f6-3da7-4489-a984-8edb564160ee None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:17.357375 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ab2f6-3da7-4489-a984-8edb564160ee None admin] PATH_INFO: `/v3/users/2d249e51ac5f4a4fb3cdc21010bd3d7c` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:17.358689 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2ab2f6-3da7-4489-a984-8edb564160ee None admin] RBAC: Authorizing `identity:get_user(user_id=2d249e51ac5f4a4fb3cdc21010bd3d7c)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:17.360169 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2ab2f6-3da7-4489-a984-8edb564160ee None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:17.364208 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980bf864-3a88-4438-b0b7-df7ef32f0ed1 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:17.366587 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 166/329] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:09:17 2022] GET /identity/v3/users/2d249e51ac5f4a4fb3cdc21010bd3d7c => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:17.369531 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2ab2f6-3da7-4489-a984-8edb564160ee None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:17.488559 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2ab2f6-3da7-4489-a984-8edb564160ee None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:17.525416 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dac014-7399-45e2-9dd7-37cd52cde12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:17.526435 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dac014-7399-45e2-9dd7-37cd52cde12d None admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:17.526714 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dac014-7399-45e2-9dd7-37cd52cde12d None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:17.526980 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dac014-7399-45e2-9dd7-37cd52cde12d None admin] PATH_INFO: `/v3/system/users/2d249e51ac5f4a4fb3cdc21010bd3d7c/roles/6479235705f74061aff23df5c9f89e33` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:17.528579 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dac014-7399-45e2-9dd7-37cd52cde12d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2d249e51ac5f4a4fb3cdc21010bd3d7c, role_id=6479235705f74061aff23df5c9f89e33)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:17.530144 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dac014-7399-45e2-9dd7-37cd52cde12d None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:17.549316 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 167/330] 10.222.0.50 () {58 vars in 1334 bytes} [Fri May 27 03:09:17 2022] PUT /identity/v3/system/users/2d249e51ac5f4a4fb3cdc21010bd3d7c/roles/6479235705f74061aff23df5c9f89e33 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:09:17.594041 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980bf864-3a88-4438-b0b7-df7ef32f0ed1 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:17.634246 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d005bf-aa9f-4194-9027-54e4de4bea06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:17.635074 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d005bf-aa9f-4194-9027-54e4de4bea06 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:17.635394 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d005bf-aa9f-4194-9027-54e4de4bea06 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:17.635665 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d005bf-aa9f-4194-9027-54e4de4bea06 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:17.649005 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d005bf-aa9f-4194-9027-54e4de4bea06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:17.657183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-d5d005bf-aa9f-4194-9027-54e4de4bea06 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:09:17.657183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:17.657183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:17.657183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:17.657183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:17.657183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:17.657183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:17.657183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:17.657183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:17.657183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:17.657183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:17.657183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:17.657183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:17.657183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:17.657183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:17.657183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:17.658147 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:17.658147 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:17.658147 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:17.658147 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:17.658147 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:17.658147 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:17.658147 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:17.658147 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:17.658147 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:17.658147 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:17.658147 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:17.658147 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:17.658147 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:17.658147 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:17.658147 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:17.658147 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:17.658906 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:17.658906 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:17.658906 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:17.658906 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:17.658906 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:17.658906 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:17.658906 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:17.658906 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:17.658906 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:17.658906 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:17.658906 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:09:17.658906 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:17.658906 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 164/331] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:09:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:17.667010 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dac014-7399-45e2-9dd7-37cd52cde12d None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:17.729996 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:17.800883 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dac014-7399-45e2-9dd7-37cd52cde12d None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:17.838659 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202e278-82c9-4a96-9b28-cb573a9b40b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:17.839400 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202e278-82c9-4a96-9b28-cb573a9b40b1 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:17.839679 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202e278-82c9-4a96-9b28-cb573a9b40b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:17.839946 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202e278-82c9-4a96-9b28-cb573a9b40b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:17.840739 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202e278-82c9-4a96-9b28-cb573a9b40b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:17.841069 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202e278-82c9-4a96-9b28-cb573a9b40b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:17.842477 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202e278-82c9-4a96-9b28-cb573a9b40b1 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:17.857893 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d005bf-aa9f-4194-9027-54e4de4bea06 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:17.867971 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 168/332] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 03:09:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:17.919105 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:17.984834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d005bf-aa9f-4194-9027-54e4de4bea06 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:18.024361 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a7f8da-ff68-4724-8119-2ca9555e61a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:18.025447 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a7f8da-ff68-4724-8119-2ca9555e61a2 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:18.025726 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a7f8da-ff68-4724-8119-2ca9555e61a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:18.026001 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a7f8da-ff68-4724-8119-2ca9555e61a2 None admin] PATH_INFO: `/v3/users/156f94969c9a47b997490cb92ceb4953` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:18.027217 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a7f8da-ff68-4724-8119-2ca9555e61a2 None admin] RBAC: Authorizing `identity:get_user(user_id=156f94969c9a47b997490cb92ceb4953)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:18.028676 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a7f8da-ff68-4724-8119-2ca9555e61a2 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:18.030335 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 165/333] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:09:17 2022] GET /identity/v3/users/156f94969c9a47b997490cb92ceb4953 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:18.036611 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202e278-82c9-4a96-9b28-cb573a9b40b1 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:18.151719 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202e278-82c9-4a96-9b28-cb573a9b40b1 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:18.169604 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:18.194904 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e65ffb-c7aa-4c0e-a5ef-be35950b2524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:18.195575 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e65ffb-c7aa-4c0e-a5ef-be35950b2524 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:18.195850 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e65ffb-c7aa-4c0e-a5ef-be35950b2524 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:18.196130 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e65ffb-c7aa-4c0e-a5ef-be35950b2524 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:18.196977 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e65ffb-c7aa-4c0e-a5ef-be35950b2524 None admin] RBAC: Adding query filter params (role.id=0129dd45ff5449f9957e8efdd01a55ed, user.id=156f94969c9a47b997490cb92ceb4953, scope.system=all) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:18.197280 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e65ffb-c7aa-4c0e-a5ef-be35950b2524 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:18.198734 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e65ffb-c7aa-4c0e-a5ef-be35950b2524 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:18.215299 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 169/334] 10.222.0.50 () {56 vars in 1308 bytes} [Fri May 27 03:09:18 2022] GET /identity/v3/role_assignments?user.id=156f94969c9a47b997490cb92ceb4953&scope.system=all&role.id=0129dd45ff5449f9957e8efdd01a55ed => generated 510 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:19.418986 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec18001-a88a-44a0-b975-28ead368ed7b None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:19.419537 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec18001-a88a-44a0-b975-28ead368ed7b None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:19.419537 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec18001-a88a-44a0-b975-28ead368ed7b None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:19.420892 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 166/335] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:19.432607 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e37a008-9ed4-4fe8-bf7a-947ba60d801c None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:19.432607 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e37a008-9ed4-4fe8-bf7a-947ba60d801c None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:19.432607 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e37a008-9ed4-4fe8-bf7a-947ba60d801c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:19.452417 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-9e37a008-9ed4-4fe8-bf7a-947ba60d801c None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:19.453660 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-9e37a008-9ed4-4fe8-bf7a-947ba60d801c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:19.549215 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 170/336] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:19.590676 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec18001-a88a-44a0-b975-28ead368ed7b None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:19.718645 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec18001-a88a-44a0-b975-28ead368ed7b None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:19.730268 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:19.766546 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada816de-05d8-4eaf-bd86-767ff9c4a52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:19.767377 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada816de-05d8-4eaf-bd86-767ff9c4a52b None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:19.767655 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada816de-05d8-4eaf-bd86-767ff9c4a52b None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:19.767923 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada816de-05d8-4eaf-bd86-767ff9c4a52b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:19.782318 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada816de-05d8-4eaf-bd86-767ff9c4a52b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:19.791082 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-ada816de-05d8-4eaf-bd86-767ff9c4a52b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:09:19.791082 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:19.791082 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:19.791082 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:19.791082 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:19.791082 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:19.791082 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:19.791082 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:19.791082 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:19.791082 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:19.791082 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:19.791082 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:19.791082 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:19.791082 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:19.791082 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:19.791082 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:19.792114 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:19.792114 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:19.792114 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:19.792114 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:19.792114 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:19.792114 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:19.792114 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:19.792114 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:19.792114 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:19.792114 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:19.792114 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:19.792114 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:19.792114 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:19.792114 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:19.792114 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:19.792114 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:19.792880 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:19.792880 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:19.792880 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:19.792880 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:19.792880 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:19.792880 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:19.792880 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:19.792880 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:19.792880 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:19.792880 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:19.792880 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:09:19.792880 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:19.792880 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 167/337] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 03:09:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:19.800283 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e37a008-9ed4-4fe8-bf7a-947ba60d801c None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:19.919505 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:19.922677 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e37a008-9ed4-4fe8-bf7a-947ba60d801c None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:19.961046 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a02969-f9b6-48db-9c00-b1d1e601d13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:19.961720 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a02969-f9b6-48db-9c00-b1d1e601d13a None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:19.961995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a02969-f9b6-48db-9c00-b1d1e601d13a None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:19.962267 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a02969-f9b6-48db-9c00-b1d1e601d13a None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:19.963015 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a02969-f9b6-48db-9c00-b1d1e601d13a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:19.963312 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a02969-f9b6-48db-9c00-b1d1e601d13a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:19.964858 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a02969-f9b6-48db-9c00-b1d1e601d13a None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:19.975561 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 171/338] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:09:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:19.980166 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada816de-05d8-4eaf-bd86-767ff9c4a52b None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:20.107438 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada816de-05d8-4eaf-bd86-767ff9c4a52b None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:20.147239 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d418cbc4-058e-4d16-bf11-58d62b0ca0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:20.148296 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418cbc4-058e-4d16-bf11-58d62b0ca0e2 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:20.148745 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418cbc4-058e-4d16-bf11-58d62b0ca0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:20.149161 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d418cbc4-058e-4d16-bf11-58d62b0ca0e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:20.158067 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99abb39-8a00-46c9-a269-b7d5d7fdb12d None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:20.158382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99abb39-8a00-46c9-a269-b7d5d7fdb12d None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:20.158612 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99abb39-8a00-46c9-a269-b7d5d7fdb12d None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:20.159985 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 172/339] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:09:20.174624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6b11d-7dc4-4bfa-b692-2b2634c7a35f None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:20.175219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6b11d-7dc4-4bfa-b692-2b2634c7a35f None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:20.175715 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6b11d-7dc4-4bfa-b692-2b2634c7a35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:20.208163 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:20.210762 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-71c6b11d-7dc4-4bfa-b692-2b2634c7a35f None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:20.212180 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-71c6b11d-7dc4-4bfa-b692-2b2634c7a35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:20.242460 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d418cbc4-058e-4d16-bf11-58d62b0ca0e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:20.244087 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d418cbc4-058e-4d16-bf11-58d62b0ca0e2 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:20.350514 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 173/340] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:20.352338 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d018b-c2b1-410a-97f4-9377375d6dd7 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:20.353026 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d018b-c2b1-410a-97f4-9377375d6dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:20.354689 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727d018b-c2b1-410a-97f4-9377375d6dd7 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:20.357233 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 174/341] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:20.361676 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd99b69-ab13-4544-ac7a-cd6cc6d1290e None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:20.362179 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd99b69-ab13-4544-ac7a-cd6cc6d1290e None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:20.362641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd99b69-ab13-4544-ac7a-cd6cc6d1290e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:20.390102 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-6fd99b69-ab13-4544-ac7a-cd6cc6d1290e None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:20.392327 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-6fd99b69-ab13-4544-ac7a-cd6cc6d1290e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:20.431854 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 168/342] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:09:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:09:20.433416 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d418cbc4-058e-4d16-bf11-58d62b0ca0e2 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:20.524740 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 175/343] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:20.526194 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd99b69-ab13-4544-ac7a-cd6cc6d1290e None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:20.558854 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:20.597309 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:20.658520 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd99b69-ab13-4544-ac7a-cd6cc6d1290e None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:20.671991 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d418cbc4-058e-4d16-bf11-58d62b0ca0e2 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:20.705262 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee8acd-53d3-4594-afb7-bca5622f37c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:20.706627 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee8acd-53d3-4594-afb7-bca5622f37c7 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:20.707241 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee8acd-53d3-4594-afb7-bca5622f37c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:20.707650 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee8acd-53d3-4594-afb7-bca5622f37c7 None admin] PATH_INFO: `/v3/projects/4edef8e255584377bf94a9ac9712b4b7` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:20.709155 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee8acd-53d3-4594-afb7-bca5622f37c7 None admin] RBAC: Authorizing `identity:get_project(project_id=4edef8e255584377bf94a9ac9712b4b7)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:20.712931 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee8acd-53d3-4594-afb7-bca5622f37c7 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:20.715239 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 176/344] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:09:20 2022] GET /identity/v3/projects/4edef8e255584377bf94a9ac9712b4b7 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:20.716181 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee8acd-53d3-4594-afb7-bca5622f37c7 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:20.744178 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a417804-e5fd-4a1b-9bff-7ca4cfb2a853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:20.745262 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a417804-e5fd-4a1b-9bff-7ca4cfb2a853 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:20.745698 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a417804-e5fd-4a1b-9bff-7ca4cfb2a853 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:20.746150 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a417804-e5fd-4a1b-9bff-7ca4cfb2a853 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:20.758428 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a417804-e5fd-4a1b-9bff-7ca4cfb2a853 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:20.778498 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-1a417804-e5fd-4a1b-9bff-7ca4cfb2a853 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:09:20.778498 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:20.778498 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:20.778498 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:20.778498 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:20.778498 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:20.778498 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:20.778498 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:20.778498 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:20.778498 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:20.778498 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:20.778498 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:20.778498 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:20.778498 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:20.778498 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:20.778498 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:20.779646 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:20.779646 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:20.779646 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:20.779646 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:20.779646 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:20.779646 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:20.779646 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:20.779646 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:20.779646 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:20.779646 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:20.779646 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:20.779646 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:20.779646 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:20.779646 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:20.779646 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:20.779646 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:20.780616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:20.780616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:20.780616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:20.780616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:20.780616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:20.780616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:20.780616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:20.780616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:20.780616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:20.780616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:20.780616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:09:20.780616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:20.783661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 169/345] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:09:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:20.789934 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a417804-e5fd-4a1b-9bff-7ca4cfb2a853 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:20.856555 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:20.868367 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee8acd-53d3-4594-afb7-bca5622f37c7 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:20.908012 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4bdcd8-d073-4d0c-86a0-e5e01fcb650d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:20.909201 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4bdcd8-d073-4d0c-86a0-e5e01fcb650d None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:20.909616 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4bdcd8-d073-4d0c-86a0-e5e01fcb650d None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:20.910355 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4bdcd8-d073-4d0c-86a0-e5e01fcb650d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:20.922433 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4bdcd8-d073-4d0c-86a0-e5e01fcb650d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:20.931098 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-8b4bdcd8-d073-4d0c-86a0-e5e01fcb650d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:09:20.931098 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:20.931098 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:20.931098 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:20.931098 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:20.931098 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:20.931098 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:20.931098 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:20.931098 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:20.931098 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:20.931098 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:20.931098 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:20.931098 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:20.931098 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:20.931098 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:20.931098 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:20.932249 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:20.932249 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:20.932249 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:20.932249 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:20.932249 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:20.932249 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:20.932249 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:20.932249 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:20.932249 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:20.932249 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:20.932249 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:20.932249 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:20.932249 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:20.932249 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:20.932249 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:20.932249 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:20.933311 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:20.933311 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:20.933311 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:20.933311 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:20.933311 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:20.933311 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:20.933311 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:20.933311 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:20.933311 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:20.933311 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:20.933311 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:09:20.933311 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:20.935454 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 177/346] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:09:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:20.936384 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4bdcd8-d073-4d0c-86a0-e5e01fcb650d None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:20.973908 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a417804-e5fd-4a1b-9bff-7ca4cfb2a853 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:21.046744 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:21.049565 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4bdcd8-d073-4d0c-86a0-e5e01fcb650d None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:21.066758 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d867a176-14ce-4005-8cfd-e6ae98953ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:21.067344 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d867a176-14ce-4005-8cfd-e6ae98953ea8 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:21.067626 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d867a176-14ce-4005-8cfd-e6ae98953ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:21.067906 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d867a176-14ce-4005-8cfd-e6ae98953ea8 None admin] PATH_INFO: `/v3/groups/fa3d562d2df749e18edd0819748e3e90` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:21.086085 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d867a176-14ce-4005-8cfd-e6ae98953ea8 None admin] RBAC: Authorizing `identity:get_group(group_id=fa3d562d2df749e18edd0819748e3e90)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:21.087365 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d867a176-14ce-4005-8cfd-e6ae98953ea8 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:21.089560 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 170/347] 10.222.0.50 () {56 vars in 1180 bytes} [Fri May 27 03:09:20 2022] GET /identity/v3/groups/fa3d562d2df749e18edd0819748e3e90 => generated 226 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:21.090407 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d867a176-14ce-4005-8cfd-e6ae98953ea8 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:21.091852 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f9f47-d429-4c93-8790-450fab50726a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:21.092810 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f9f47-d429-4c93-8790-450fab50726a None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:21.093260 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f9f47-d429-4c93-8790-450fab50726a None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:21.093758 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f9f47-d429-4c93-8790-450fab50726a None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:21.094663 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f9f47-d429-4c93-8790-450fab50726a None admin] RBAC: Adding query filter params (name=member) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:21.095099 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f9f47-d429-4c93-8790-450fab50726a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:21.096751 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716f9f47-d429-4c93-8790-450fab50726a None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:21.106976 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 178/348] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 03:09:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:21.108110 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f9f47-d429-4c93-8790-450fab50726a None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:21.130933 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:21.217731 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f9f47-d429-4c93-8790-450fab50726a None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:21.257992 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62720ad-3675-4795-bbc8-271d36d5d786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:21.259068 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62720ad-3675-4795-bbc8-271d36d5d786 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:21.259528 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62720ad-3675-4795-bbc8-271d36d5d786 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:21.259929 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62720ad-3675-4795-bbc8-271d36d5d786 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:21.261025 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62720ad-3675-4795-bbc8-271d36d5d786 None admin] RBAC: Adding query filter params (group.id=fa3d562d2df749e18edd0819748e3e90, role.id=9e63b73b3ce24bb3ad214ecdb6132696, scope.project.id=4edef8e255584377bf94a9ac9712b4b7) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:21.261494 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62720ad-3675-4795-bbc8-271d36d5d786 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:21.263236 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62720ad-3675-4795-bbc8-271d36d5d786 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:21.264819 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d867a176-14ce-4005-8cfd-e6ae98953ea8 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:21.270203 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 179/349] 10.222.0.50 () {56 vars in 1376 bytes} [Fri May 27 03:09:21 2022] GET /identity/v3/role_assignments?group.id=fa3d562d2df749e18edd0819748e3e90&scope.project.id=4edef8e255584377bf94a9ac9712b4b7&role.id=9e63b73b3ce24bb3ad214ecdb6132696 => generated 260 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:21.271441 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62720ad-3675-4795-bbc8-271d36d5d786 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:21.323746 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea74f5-55fe-4c30-878c-cab52af879ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:21.324583 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea74f5-55fe-4c30-878c-cab52af879ca None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:21.324870 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea74f5-55fe-4c30-878c-cab52af879ca None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:21.325196 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea74f5-55fe-4c30-878c-cab52af879ca None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:21.325976 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea74f5-55fe-4c30-878c-cab52af879ca None admin] RBAC: Adding query filter params (name=member) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:21.326285 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea74f5-55fe-4c30-878c-cab52af879ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:21.328101 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea74f5-55fe-4c30-878c-cab52af879ca None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:21.338950 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 171/350] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 03:09:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:21.340470 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75446cb6-902a-48aa-9284-9d199f4dbcb3 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:21.340764 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75446cb6-902a-48aa-9284-9d199f4dbcb3 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:21.341070 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75446cb6-902a-48aa-9284-9d199f4dbcb3 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:21.342598 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 172/351] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:21.346137 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75446cb6-902a-48aa-9284-9d199f4dbcb3 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:21.422328 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:21.446856 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:21.475031 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62720ad-3675-4795-bbc8-271d36d5d786 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:21.481052 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75446cb6-902a-48aa-9284-9d199f4dbcb3 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:21.517686 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c938fbe1-b03e-4252-94b0-50c518794e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:21.518352 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938fbe1-b03e-4252-94b0-50c518794e64 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:21.518698 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938fbe1-b03e-4252-94b0-50c518794e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:21.518961 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938fbe1-b03e-4252-94b0-50c518794e64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:21.528409 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3451dc-3dc4-4b8a-9fb6-6cc5779ee802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:21.529485 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3451dc-3dc4-4b8a-9fb6-6cc5779ee802 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:21.529965 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3451dc-3dc4-4b8a-9fb6-6cc5779ee802 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:21.530432 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3451dc-3dc4-4b8a-9fb6-6cc5779ee802 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:21.611621 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c938fbe1-b03e-4252-94b0-50c518794e64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:21.616063 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c938fbe1-b03e-4252-94b0-50c518794e64 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:21.628095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3451dc-3dc4-4b8a-9fb6-6cc5779ee802 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:21.629629 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3451dc-3dc4-4b8a-9fb6-6cc5779ee802 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:21.699805 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:21.719701 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:21.750038 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 180/352] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:09:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:09:21.752941 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba29c5-3006-4451-b1d5-d849f504ac5d None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:21.753288 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba29c5-3006-4451-b1d5-d849f504ac5d None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:21.753557 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ba29c5-3006-4451-b1d5-d849f504ac5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:21.762359 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 173/353] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:09:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:09:21.763764 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a996f66-868a-453c-8e68-e6e1777f18b7 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:21.764063 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a996f66-868a-453c-8e68-e6e1777f18b7 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:21.764333 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a996f66-868a-453c-8e68-e6e1777f18b7 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:21.765991 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 174/354] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:21.767199 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa026d0e-20ca-41e3-9ade-5dccc6bc8d05 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:21.767482 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa026d0e-20ca-41e3-9ade-5dccc6bc8d05 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:21.767754 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa026d0e-20ca-41e3-9ade-5dccc6bc8d05 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:21.773584 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 175/355] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:21.774430 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa026d0e-20ca-41e3-9ade-5dccc6bc8d05 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:21.783032 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-d0ba29c5-3006-4451-b1d5-d849f504ac5d None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:21.784198 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-d0ba29c5-3006-4451-b1d5-d849f504ac5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:21.874122 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 181/356] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:21.875074 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba29c5-3006-4451-b1d5-d849f504ac5d None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:21.895736 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa026d0e-20ca-41e3-9ade-5dccc6bc8d05 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:21.935791 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bffb7d0-7db0-4f2e-aa5f-c3faf80db8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:21.936909 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffb7d0-7db0-4f2e-aa5f-c3faf80db8de None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:21.937600 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffb7d0-7db0-4f2e-aa5f-c3faf80db8de None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:21.937881 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffb7d0-7db0-4f2e-aa5f-c3faf80db8de None admin] PATH_INFO: `/v3/projects/4edef8e255584377bf94a9ac9712b4b7` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:21.939062 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bffb7d0-7db0-4f2e-aa5f-c3faf80db8de None admin] RBAC: Authorizing `identity:get_project(project_id=4edef8e255584377bf94a9ac9712b4b7)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:21.941128 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bffb7d0-7db0-4f2e-aa5f-c3faf80db8de None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:21.943715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 176/357] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:09:21 2022] GET /identity/v3/projects/4edef8e255584377bf94a9ac9712b4b7 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:21.945377 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe5385-d758-40be-b6dd-883ee9620e62 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:21.945740 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe5385-d758-40be-b6dd-883ee9620e62 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:21.945927 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe5385-d758-40be-b6dd-883ee9620e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:21.964090 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-3dfe5385-d758-40be-b6dd-883ee9620e62 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:21.965312 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-3dfe5385-d758-40be-b6dd-883ee9620e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:21.987852 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:22.000217 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ba29c5-3006-4451-b1d5-d849f504ac5d None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:22.013412 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:22.054343 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cd8904-7a05-4f8b-911b-caddfa8b735f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:22.055232 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cd8904-7a05-4f8b-911b-caddfa8b735f None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:22.055512 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cd8904-7a05-4f8b-911b-caddfa8b735f None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:22.055800 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cd8904-7a05-4f8b-911b-caddfa8b735f None admin] PATH_INFO: `/v3/projects/5e327e3411214877a9f427330ef40dc5` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:22.056941 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cd8904-7a05-4f8b-911b-caddfa8b735f None admin] RBAC: Authorizing `identity:get_project(project_id=5e327e3411214877a9f427330ef40dc5)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:22.057576 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 177/358] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:22.058422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cd8904-7a05-4f8b-911b-caddfa8b735f None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:22.058648 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545368b7-cb4f-42bf-95d0-476849ac069b None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:22.058930 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545368b7-cb4f-42bf-95d0-476849ac069b None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:22.059199 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545368b7-cb4f-42bf-95d0-476849ac069b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:22.060260 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 182/359] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:09:21 2022] GET /identity/v3/projects/5e327e3411214877a9f427330ef40dc5 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:22.060868 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cd8904-7a05-4f8b-911b-caddfa8b735f None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:22.082220 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-545368b7-cb4f-42bf-95d0-476849ac069b None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:22.084299 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-545368b7-cb4f-42bf-95d0-476849ac069b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:22.173248 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 178/360] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:22.174287 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545368b7-cb4f-42bf-95d0-476849ac069b None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:22.200639 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cd8904-7a05-4f8b-911b-caddfa8b735f None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:22.252189 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cf3108-723c-4504-87c8-77d06992a8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:22.257008 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf3108-723c-4504-87c8-77d06992a8f8 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:22.257008 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf3108-723c-4504-87c8-77d06992a8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:22.257008 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf3108-723c-4504-87c8-77d06992a8f8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:22.273477 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cf3108-723c-4504-87c8-77d06992a8f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:22.281797 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-12cf3108-723c-4504-87c8-77d06992a8f8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:09:22.281797 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:22.281797 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:22.281797 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:22.281797 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:22.281797 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:22.281797 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:22.281797 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:22.281797 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:22.281797 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:22.281797 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:22.281797 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:22.281797 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:22.281797 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:22.281797 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:22.281797 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:22.282672 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:22.282672 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:22.282672 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:22.282672 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:22.282672 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:22.282672 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:22.282672 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:22.282672 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:22.282672 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:22.282672 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:22.282672 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:22.282672 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:22.282672 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:22.282672 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:22.282672 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:22.282672 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:22.283433 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:22.283433 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:22.283433 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:22.283433 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:22.283433 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:22.283433 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:22.283433 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:22.283433 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:22.283433 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:22.283433 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:22.283433 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:09:22.283433 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:22.283433 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 183/361] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 03:09:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:22.285725 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cf3108-723c-4504-87c8-77d06992a8f8 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:22.313664 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:22.329008 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:22.339153 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545368b7-cb4f-42bf-95d0-476849ac069b None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:22.386091 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd205feb-ed68-497d-9d25-c3a4a65bb4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:22.387741 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd205feb-ed68-497d-9d25-c3a4a65bb4fb None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:22.388444 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd205feb-ed68-497d-9d25-c3a4a65bb4fb None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:22.388925 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd205feb-ed68-497d-9d25-c3a4a65bb4fb None admin] PATH_INFO: `/v3/groups/fa3d562d2df749e18edd0819748e3e90` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:22.391000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd205feb-ed68-497d-9d25-c3a4a65bb4fb None admin] RBAC: Authorizing `identity:get_group(group_id=fa3d562d2df749e18edd0819748e3e90)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:22.392642 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd205feb-ed68-497d-9d25-c3a4a65bb4fb None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:22.395648 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 179/362] 10.222.0.50 () {56 vars in 1180 bytes} [Fri May 27 03:09:22 2022] GET /identity/v3/groups/fa3d562d2df749e18edd0819748e3e90 => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:22.396682 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd205feb-ed68-497d-9d25-c3a4a65bb4fb None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:22.408167 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cf3108-723c-4504-87c8-77d06992a8f8 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:22.449262 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c822593-e4d4-4356-a7de-d65b4616d4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:22.450143 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c822593-e4d4-4356-a7de-d65b4616d4ec None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:22.450449 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c822593-e4d4-4356-a7de-d65b4616d4ec None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:22.450800 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c822593-e4d4-4356-a7de-d65b4616d4ec None admin] PATH_INFO: `/v3/groups/fa3d562d2df749e18edd0819748e3e90` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:22.452083 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c822593-e4d4-4356-a7de-d65b4616d4ec None admin] RBAC: Authorizing `identity:get_group(group_id=fa3d562d2df749e18edd0819748e3e90)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:22.453957 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c822593-e4d4-4356-a7de-d65b4616d4ec None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:22.455675 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 184/363] 10.222.0.50 () {56 vars in 1180 bytes} [Fri May 27 03:09:22 2022] GET /identity/v3/groups/fa3d562d2df749e18edd0819748e3e90 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:22.456450 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c822593-e4d4-4356-a7de-d65b4616d4ec None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:22.529607 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd205feb-ed68-497d-9d25-c3a4a65bb4fb None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:22.582285 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaa10a4-3d8a-40e5-8c12-cd261ec11ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:22.583172 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa10a4-3d8a-40e5-8c12-cd261ec11ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:22.583449 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa10a4-3d8a-40e5-8c12-cd261ec11ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:22.583754 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa10a4-3d8a-40e5-8c12-cd261ec11ef1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:22.589896 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c822593-e4d4-4356-a7de-d65b4616d4ec None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:22.591509 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:22.599020 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaa10a4-3d8a-40e5-8c12-cd261ec11ef1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:22.606835 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-4eaa10a4-3d8a-40e5-8c12-cd261ec11ef1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:09:22.606835 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:22.606835 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:22.606835 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:22.606835 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:22.606835 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:22.606835 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:22.606835 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:22.606835 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:22.606835 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:22.606835 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:22.606835 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:22.606835 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:22.606835 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:22.606835 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:22.606835 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:22.607749 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:22.607749 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:22.607749 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:22.607749 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:22.607749 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:22.607749 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:22.607749 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:22.607749 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:22.607749 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:22.607749 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:22.607749 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:22.607749 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:22.607749 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:22.607749 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:22.607749 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:22.607749 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:22.608525 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:22.608525 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:22.608525 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:22.608525 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:22.608525 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:22.608525 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:22.608525 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:22.608525 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:22.608525 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:22.608525 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:22.608525 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 03:09:22.608525 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:22.608525 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 180/364] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:09:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:22.609686 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaa10a4-3d8a-40e5-8c12-cd261ec11ef1 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:22.625819 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:22.637518 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7011faa2-fe4b-4b26-9ff1-fe96e9a22eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:22.638936 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011faa2-fe4b-4b26-9ff1-fe96e9a22eb7 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:22.639214 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011faa2-fe4b-4b26-9ff1-fe96e9a22eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:22.639494 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011faa2-fe4b-4b26-9ff1-fe96e9a22eb7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:22.652782 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7011faa2-fe4b-4b26-9ff1-fe96e9a22eb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:22.660764 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-7011faa2-fe4b-4b26-9ff1-fe96e9a22eb7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:09:22.660764 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:22.660764 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:22.660764 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:22.660764 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:22.660764 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:22.660764 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:22.660764 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:22.660764 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:22.660764 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:22.660764 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:22.660764 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:22.660764 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:22.660764 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:22.660764 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:22.660764 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:22.661960 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:22.661960 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:22.661960 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:22.661960 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:22.661960 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:22.661960 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:22.661960 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:22.661960 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:22.661960 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:22.661960 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:22.661960 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:22.661960 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:22.661960 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:22.661960 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:22.661960 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:22.661960 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:22.662948 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:22.662948 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:22.662948 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:22.662948 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:22.662948 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:22.662948 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:22.662948 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:22.662948 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:22.662948 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:22.662948 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:22.662948 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:09:22.662948 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:22.662948 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 185/365] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 03:09:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:22.662948 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7011faa2-fe4b-4b26-9ff1-fe96e9a22eb7 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:22.750986 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaa10a4-3d8a-40e5-8c12-cd261ec11ef1 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:22.791585 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7011faa2-fe4b-4b26-9ff1-fe96e9a22eb7 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:22.791758 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a4b3b-dae1-4c00-8362-baf2143fd171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:22.792282 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a4b3b-dae1-4c00-8362-baf2143fd171 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:22.792561 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a4b3b-dae1-4c00-8362-baf2143fd171 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:22.792832 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a4b3b-dae1-4c00-8362-baf2143fd171 None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:22.794102 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a4b3b-dae1-4c00-8362-baf2143fd171 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:22.794499 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a4b3b-dae1-4c00-8362-baf2143fd171 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:22.795991 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343a4b3b-dae1-4c00-8362-baf2143fd171 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:22.805861 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 181/366] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:09:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:22.806521 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a4b3b-dae1-4c00-8362-baf2143fd171 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:22.829593 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e973a121-ebd4-4084-baab-aa99ce1d6e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:22.830629 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e973a121-ebd4-4084-baab-aa99ce1d6e86 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:22.831019 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e973a121-ebd4-4084-baab-aa99ce1d6e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:22.831301 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e973a121-ebd4-4084-baab-aa99ce1d6e86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:22.832246 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e973a121-ebd4-4084-baab-aa99ce1d6e86 None admin] RBAC: Adding query filter params (group.id=fa3d562d2df749e18edd0819748e3e90, role.id=0129dd45ff5449f9957e8efdd01a55ed, scope.project.id=4edef8e255584377bf94a9ac9712b4b7) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:22.832556 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e973a121-ebd4-4084-baab-aa99ce1d6e86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:22.834648 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e973a121-ebd4-4084-baab-aa99ce1d6e86 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:22.843964 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 186/367] 10.222.0.50 () {56 vars in 1376 bytes} [Fri May 27 03:09:22 2022] GET /identity/v3/role_assignments?group.id=fa3d562d2df749e18edd0819748e3e90&scope.project.id=4edef8e255584377bf94a9ac9712b4b7&role.id=0129dd45ff5449f9957e8efdd01a55ed => generated 260 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:22.845246 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e973a121-ebd4-4084-baab-aa99ce1d6e86 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:22.888887 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:22.907875 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:22.939026 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a4b3b-dae1-4c00-8362-baf2143fd171 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:22.976720 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e973a121-ebd4-4084-baab-aa99ce1d6e86 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:22.978327 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e8f41-3a9f-4a6d-b909-55f4edc5f83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:22.979136 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e8f41-3a9f-4a6d-b909-55f4edc5f83d None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:22.979423 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e8f41-3a9f-4a6d-b909-55f4edc5f83d None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:22.979697 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e8f41-3a9f-4a6d-b909-55f4edc5f83d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:22.980689 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05e8f41-3a9f-4a6d-b909-55f4edc5f83d None admin] RBAC: Adding query filter params (group.id=fa3d562d2df749e18edd0819748e3e90, role.id=0129dd45ff5449f9957e8efdd01a55ed, scope.project.id=5e327e3411214877a9f427330ef40dc5) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:22.981985 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05e8f41-3a9f-4a6d-b909-55f4edc5f83d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:22.982661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05e8f41-3a9f-4a6d-b909-55f4edc5f83d None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:22.989287 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 182/368] 10.222.0.50 () {56 vars in 1376 bytes} [Fri May 27 03:09:22 2022] GET /identity/v3/role_assignments?group.id=fa3d562d2df749e18edd0819748e3e90&scope.project.id=5e327e3411214877a9f427330ef40dc5&role.id=0129dd45ff5449f9957e8efdd01a55ed => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:22.990397 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e8f41-3a9f-4a6d-b909-55f4edc5f83d None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:23.023260 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92459838-4026-4650-8556-3f66b5274287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:23.024398 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92459838-4026-4650-8556-3f66b5274287 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:23.024986 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92459838-4026-4650-8556-3f66b5274287 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:23.025430 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92459838-4026-4650-8556-3f66b5274287 None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:23.026364 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92459838-4026-4650-8556-3f66b5274287 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:23.026843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92459838-4026-4650-8556-3f66b5274287 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:23.028747 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92459838-4026-4650-8556-3f66b5274287 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:23.040620 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 187/369] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 03:09:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:23.041959 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92459838-4026-4650-8556-3f66b5274287 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:23.124810 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e8f41-3a9f-4a6d-b909-55f4edc5f83d None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:23.154772 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:23.170610 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92459838-4026-4650-8556-3f66b5274287 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:23.175057 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7418a-20af-49e1-a840-ef220108ccea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:23.175755 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7418a-20af-49e1-a840-ef220108ccea None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:23.176038 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7418a-20af-49e1-a840-ef220108ccea None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:23.176322 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7418a-20af-49e1-a840-ef220108ccea None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:23.177126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c7418a-20af-49e1-a840-ef220108ccea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:23.177505 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c7418a-20af-49e1-a840-ef220108ccea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:23.179132 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c7418a-20af-49e1-a840-ef220108ccea None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:23.187250 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:23.198074 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 183/370] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:09:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:23.199330 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7418a-20af-49e1-a840-ef220108ccea None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:23.223879 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc0b317-1d5f-4977-a5ab-cd2df3bc0336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:23.225174 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0b317-1d5f-4977-a5ab-cd2df3bc0336 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:23.225892 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0b317-1d5f-4977-a5ab-cd2df3bc0336 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:23.226186 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc0b317-1d5f-4977-a5ab-cd2df3bc0336 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:23.326885 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc0b317-1d5f-4977-a5ab-cd2df3bc0336 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:23.329003 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc0b317-1d5f-4977-a5ab-cd2df3bc0336 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:23.337649 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7418a-20af-49e1-a840-ef220108ccea None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:23.382939 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb9ce1-12fb-4a94-a8b0-17868e11ee11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:23.383856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb9ce1-12fb-4a94-a8b0-17868e11ee11 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:23.384511 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb9ce1-12fb-4a94-a8b0-17868e11ee11 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:23.384511 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb9ce1-12fb-4a94-a8b0-17868e11ee11 None admin] PATH_INFO: `/v3/users/2d249e51ac5f4a4fb3cdc21010bd3d7c` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:23.385825 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eb9ce1-12fb-4a94-a8b0-17868e11ee11 None admin] RBAC: Authorizing `identity:get_user(user_id=2d249e51ac5f4a4fb3cdc21010bd3d7c)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:23.387469 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eb9ce1-12fb-4a94-a8b0-17868e11ee11 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:23.389201 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 184/371] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:09:23 2022] GET /identity/v3/users/2d249e51ac5f4a4fb3cdc21010bd3d7c => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:23.389976 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb9ce1-12fb-4a94-a8b0-17868e11ee11 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:23.449687 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:23.470037 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 188/372] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:09:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:09:23.470916 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc0b317-1d5f-4977-a5ab-cd2df3bc0336 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:23.483527 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:23.544752 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb9ce1-12fb-4a94-a8b0-17868e11ee11 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:23.610049 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8286b588-8d53-4484-aa14-09ed4f011d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:23.611083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286b588-8d53-4484-aa14-09ed4f011d9d None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:23.611543 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286b588-8d53-4484-aa14-09ed4f011d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:23.611989 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8286b588-8d53-4484-aa14-09ed4f011d9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:23.651201 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc0b317-1d5f-4977-a5ab-cd2df3bc0336 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:23.702698 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8286b588-8d53-4484-aa14-09ed4f011d9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:23.704216 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8286b588-8d53-4484-aa14-09ed4f011d9d None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:23.767057 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7132fb48-036a-4d32-bc10-7e334b588a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:23.767737 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7132fb48-036a-4d32-bc10-7e334b588a10 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:23.768014 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7132fb48-036a-4d32-bc10-7e334b588a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:23.768309 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7132fb48-036a-4d32-bc10-7e334b588a10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:23.769149 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7132fb48-036a-4d32-bc10-7e334b588a10 None admin] RBAC: Adding query filter params (role.id=6479235705f74061aff23df5c9f89e33, user.id=2d249e51ac5f4a4fb3cdc21010bd3d7c, scope.system=all) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:23.769462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7132fb48-036a-4d32-bc10-7e334b588a10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:23.770942 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7132fb48-036a-4d32-bc10-7e334b588a10 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:23.789584 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:23.815350 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 189/373] 10.222.0.50 () {56 vars in 1308 bytes} [Fri May 27 03:09:23 2022] GET /identity/v3/role_assignments?user.id=2d249e51ac5f4a4fb3cdc21010bd3d7c&scope.system=all&role.id=6479235705f74061aff23df5c9f89e33 => generated 510 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:23.816290 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7132fb48-036a-4d32-bc10-7e334b588a10 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:23.817578 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 185/374] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:09:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:09:23.823868 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8286b588-8d53-4484-aa14-09ed4f011d9d None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:23.932484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:23.982526 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8286b588-8d53-4484-aa14-09ed4f011d9d None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:24.058177 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7132fb48-036a-4d32-bc10-7e334b588a10 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:24.062618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0886ac-bd18-44f0-a50a-0c2f06cdf1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:24.063514 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0886ac-bd18-44f0-a50a-0c2f06cdf1f1 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:24.063793 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0886ac-bd18-44f0-a50a-0c2f06cdf1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:24.064067 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0886ac-bd18-44f0-a50a-0c2f06cdf1f1 None admin] PATH_INFO: `/v3/projects/73644351894d46fdb8889fca96b4c614` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:24.065512 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0886ac-bd18-44f0-a50a-0c2f06cdf1f1 None admin] RBAC: Authorizing `identity:get_project(project_id=73644351894d46fdb8889fca96b4c614)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:24.066989 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0886ac-bd18-44f0-a50a-0c2f06cdf1f1 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:24.078136 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 186/375] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:09:23 2022] GET /identity/v3/projects/73644351894d46fdb8889fca96b4c614 => generated 344 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:24.085056 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0886ac-bd18-44f0-a50a-0c2f06cdf1f1 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:24.144311 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64fca5e-dc2d-42d0-b07b-26be28181737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:24.145282 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fca5e-dc2d-42d0-b07b-26be28181737 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:24.145563 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fca5e-dc2d-42d0-b07b-26be28181737 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:24.145834 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fca5e-dc2d-42d0-b07b-26be28181737 None admin] PATH_INFO: `/v3/projects/5e327e3411214877a9f427330ef40dc5` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:24.147319 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64fca5e-dc2d-42d0-b07b-26be28181737 None admin] RBAC: Authorizing `identity:get_project(project_id=5e327e3411214877a9f427330ef40dc5)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:24.148809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64fca5e-dc2d-42d0-b07b-26be28181737 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:24.153640 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:24.154303 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 190/376] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:09:23 2022] GET /identity/v3/projects/5e327e3411214877a9f427330ef40dc5 => generated 302 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:24.158709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64fca5e-dc2d-42d0-b07b-26be28181737 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:24.297498 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0886ac-bd18-44f0-a50a-0c2f06cdf1f1 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:24.306582 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64fca5e-dc2d-42d0-b07b-26be28181737 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:24.327422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:24.346454 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6176533-33b9-4230-80ae-d7693c57c432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:24.347337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6176533-33b9-4230-80ae-d7693c57c432 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:24.347614 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6176533-33b9-4230-80ae-d7693c57c432 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:24.347886 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6176533-33b9-4230-80ae-d7693c57c432 None admin] PATH_INFO: `/v3/groups/e78521d2f9ed4d93ad55d88822834e22` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:24.355958 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6176533-33b9-4230-80ae-d7693c57c432 None admin] RBAC: Authorizing `identity:get_group(group_id=e78521d2f9ed4d93ad55d88822834e22)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:24.358036 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6176533-33b9-4230-80ae-d7693c57c432 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:24.360160 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 187/377] 10.222.0.50 () {56 vars in 1180 bytes} [Fri May 27 03:09:24 2022] GET /identity/v3/groups/e78521d2f9ed4d93ad55d88822834e22 => generated 229 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:24.361729 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5ef20b-2942-4d8d-b32f-35a71591860a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:24.362609 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ef20b-2942-4d8d-b32f-35a71591860a None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:24.362887 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ef20b-2942-4d8d-b32f-35a71591860a None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:24.363156 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ef20b-2942-4d8d-b32f-35a71591860a None admin] PATH_INFO: `/v3/groups/fa3d562d2df749e18edd0819748e3e90` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:24.364336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5ef20b-2942-4d8d-b32f-35a71591860a None admin] RBAC: Authorizing `identity:get_group(group_id=fa3d562d2df749e18edd0819748e3e90)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:24.366271 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30407bcb-0edd-4c10-8939-47a73539cfa0 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:24.366559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30407bcb-0edd-4c10-8939-47a73539cfa0 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:24.366838 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30407bcb-0edd-4c10-8939-47a73539cfa0 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:24.367057 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5ef20b-2942-4d8d-b32f-35a71591860a None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:24.368473 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 188/378] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:24.369793 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30407bcb-0edd-4c10-8939-47a73539cfa0 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:24.373678 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 191/379] 10.222.0.50 () {56 vars in 1180 bytes} [Fri May 27 03:09:24 2022] GET /identity/v3/groups/fa3d562d2df749e18edd0819748e3e90 => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:24.374986 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1901bc-8fcd-46b0-af5a-9a415e8e8b60 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:24.375271 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1901bc-8fcd-46b0-af5a-9a415e8e8b60 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:24.375545 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1901bc-8fcd-46b0-af5a-9a415e8e8b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:24.401615 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-bc1901bc-8fcd-46b0-af5a-9a415e8e8b60 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:24.402824 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-bc1901bc-8fcd-46b0-af5a-9a415e8e8b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:24.492076 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 192/380] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:24.497138 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1901bc-8fcd-46b0-af5a-9a415e8e8b60 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:24.500813 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:24.506407 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30407bcb-0edd-4c10-8939-47a73539cfa0 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:24.551468 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58276c8f-a40d-4cd4-8c81-71e12b940403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:24.552205 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58276c8f-a40d-4cd4-8c81-71e12b940403 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:24.552408 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58276c8f-a40d-4cd4-8c81-71e12b940403 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:24.553515 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58276c8f-a40d-4cd4-8c81-71e12b940403 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:24.553749 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58276c8f-a40d-4cd4-8c81-71e12b940403 None admin] RBAC: Adding query filter params (group.id=e78521d2f9ed4d93ad55d88822834e22, role.id=b4c524dd7bb44e94bd3b42d86e1cdf42, scope.project.id=73644351894d46fdb8889fca96b4c614) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:24.554061 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58276c8f-a40d-4cd4-8c81-71e12b940403 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:24.556459 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58276c8f-a40d-4cd4-8c81-71e12b940403 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:24.564411 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 189/381] 10.222.0.50 () {56 vars in 1376 bytes} [Fri May 27 03:09:24 2022] GET /identity/v3/role_assignments?group.id=e78521d2f9ed4d93ad55d88822834e22&scope.project.id=73644351894d46fdb8889fca96b4c614&role.id=b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:24.565803 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58276c8f-a40d-4cd4-8c81-71e12b940403 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:24.685635 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1901bc-8fcd-46b0-af5a-9a415e8e8b60 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:24.722151 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:24.747678 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58276c8f-a40d-4cd4-8c81-71e12b940403 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:24.789722 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93050d6b-782d-4283-b883-2277343ebccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:24.790807 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93050d6b-782d-4283-b883-2277343ebccd None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:24.791288 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93050d6b-782d-4283-b883-2277343ebccd None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:24.791756 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93050d6b-782d-4283-b883-2277343ebccd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:24.792831 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93050d6b-782d-4283-b883-2277343ebccd None admin] RBAC: Adding query filter params (group.id=fa3d562d2df749e18edd0819748e3e90, role.id=9e63b73b3ce24bb3ad214ecdb6132696, scope.project.id=5e327e3411214877a9f427330ef40dc5) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:24.793405 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93050d6b-782d-4283-b883-2277343ebccd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:24.795098 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93050d6b-782d-4283-b883-2277343ebccd None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:24.805790 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b6008a-90a3-42ae-9e46-dad5cb7d292c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:24.806560 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b6008a-90a3-42ae-9e46-dad5cb7d292c None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:24.806843 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b6008a-90a3-42ae-9e46-dad5cb7d292c None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:24.807112 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b6008a-90a3-42ae-9e46-dad5cb7d292c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:24.809427 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 193/382] 10.222.0.50 () {56 vars in 1376 bytes} [Fri May 27 03:09:24 2022] GET /identity/v3/role_assignments?group.id=fa3d562d2df749e18edd0819748e3e90&scope.project.id=5e327e3411214877a9f427330ef40dc5&role.id=9e63b73b3ce24bb3ad214ecdb6132696 => generated 260 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:24.828480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b6008a-90a3-42ae-9e46-dad5cb7d292c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:24.838613 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-a1b6008a-90a3-42ae-9e46-dad5cb7d292c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:09:24.838613 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:24.838613 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:24.838613 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:24.838613 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:24.838613 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:24.838613 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:24.838613 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:24.838613 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:24.838613 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:24.838613 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:24.838613 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:24.838613 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:24.838613 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:24.838613 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:24.838613 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:24.839579 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:24.839579 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:24.839579 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:24.839579 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:24.839579 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:24.839579 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:24.839579 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:24.839579 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:24.839579 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:24.839579 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:24.839579 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:24.839579 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:24.839579 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:24.839579 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:24.839579 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:24.839579 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:24.840825 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:24.840825 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:24.840825 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:24.840825 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:24.840825 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:24.840825 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:24.840825 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:24.840825 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:24.840825 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:24.840825 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:24.840825 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:09:24.840825 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:24.840825 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 190/383] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 03:09:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:24.846411 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93050d6b-782d-4283-b883-2277343ebccd None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:24.969440 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93050d6b-782d-4283-b883-2277343ebccd None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:25.012878 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6393f7c1-c65c-48fd-8b49-23688388880f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:25.013615 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6393f7c1-c65c-48fd-8b49-23688388880f None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:25.013943 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6393f7c1-c65c-48fd-8b49-23688388880f None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:25.014222 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6393f7c1-c65c-48fd-8b49-23688388880f None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:25.014985 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6393f7c1-c65c-48fd-8b49-23688388880f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:25.015293 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6393f7c1-c65c-48fd-8b49-23688388880f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:25.016742 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6393f7c1-c65c-48fd-8b49-23688388880f None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:25.028167 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 194/384] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:09:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:25.034181 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b6008a-90a3-42ae-9e46-dad5cb7d292c None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:25.069488 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:25.155383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b6008a-90a3-42ae-9e46-dad5cb7d292c None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:25.198081 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a0ca90-ad60-442f-be91-e44431723130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:25.198954 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0ca90-ad60-442f-be91-e44431723130 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:25.199234 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0ca90-ad60-442f-be91-e44431723130 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:25.199508 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0ca90-ad60-442f-be91-e44431723130 None admin] PATH_INFO: `/v3/groups/fa3d562d2df749e18edd0819748e3e90` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:25.201639 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a0ca90-ad60-442f-be91-e44431723130 None admin] RBAC: Authorizing `identity:get_group(group_id=fa3d562d2df749e18edd0819748e3e90)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:25.203099 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a0ca90-ad60-442f-be91-e44431723130 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:25.204732 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 191/385] 10.222.0.50 () {56 vars in 1180 bytes} [Fri May 27 03:09:25 2022] GET /identity/v3/groups/fa3d562d2df749e18edd0819748e3e90 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:25.211094 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6393f7c1-c65c-48fd-8b49-23688388880f None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:25.257112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:25.337187 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6393f7c1-c65c-48fd-8b49-23688388880f None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:25.376332 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5c9579-61c8-4737-afaa-d099651dc9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:25.377256 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c9579-61c8-4737-afaa-d099651dc9e0 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:25.377533 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c9579-61c8-4737-afaa-d099651dc9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:25.377803 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5c9579-61c8-4737-afaa-d099651dc9e0 None admin] PATH_INFO: `/v3/projects/4edef8e255584377bf94a9ac9712b4b7` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:25.378985 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5c9579-61c8-4737-afaa-d099651dc9e0 None admin] RBAC: Authorizing `identity:get_project(project_id=4edef8e255584377bf94a9ac9712b4b7)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:25.380612 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5c9579-61c8-4737-afaa-d099651dc9e0 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:25.386784 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 195/386] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:09:25 2022] GET /identity/v3/projects/4edef8e255584377bf94a9ac9712b4b7 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:25.392496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a0ca90-ad60-442f-be91-e44431723130 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:25.503819 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:25.514919 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a0ca90-ad60-442f-be91-e44431723130 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:25.563977 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc1c9de-d2ad-434b-aa53-41b35c162567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:25.564874 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1c9de-d2ad-434b-aa53-41b35c162567 None admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:25.565180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1c9de-d2ad-434b-aa53-41b35c162567 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:25.565456 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc1c9de-d2ad-434b-aa53-41b35c162567 None admin] PATH_INFO: `/v3/projects/4edef8e255584377bf94a9ac9712b4b7/groups/fa3d562d2df749e18edd0819748e3e90/roles/9e63b73b3ce24bb3ad214ecdb6132696` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:25.573743 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc1c9de-d2ad-434b-aa53-41b35c162567 None admin] RBAC: Authorizing `identity:create_grant(project_id=4edef8e255584377bf94a9ac9712b4b7, group_id=fa3d562d2df749e18edd0819748e3e90, role_id=9e63b73b3ce24bb3ad214ecdb6132696)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:25.575206 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc1c9de-d2ad-434b-aa53-41b35c162567 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:25.606660 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 192/387] 10.222.0.50 () {58 vars in 1442 bytes} [Fri May 27 03:09:25 2022] PUT /identity/v3/projects/4edef8e255584377bf94a9ac9712b4b7/groups/fa3d562d2df749e18edd0819748e3e90/roles/9e63b73b3ce24bb3ad214ecdb6132696 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:09:25.813365 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea6888-040e-4c29-9182-df451f008eb7 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:25.814335 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea6888-040e-4c29-9182-df451f008eb7 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:25.814837 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea6888-040e-4c29-9182-df451f008eb7 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:25.816646 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 196/388] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:25.824582 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428349bf-73a0-452c-ab32-5bae2731770b None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:25.825205 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428349bf-73a0-452c-ab32-5bae2731770b None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:25.825680 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428349bf-73a0-452c-ab32-5bae2731770b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:25.845809 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-428349bf-73a0-452c-ab32-5bae2731770b None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:25.847378 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-428349bf-73a0-452c-ab32-5bae2731770b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:25.934588 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 193/389] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:09:25.971470 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea6888-040e-4c29-9182-df451f008eb7 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:26.118624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea6888-040e-4c29-9182-df451f008eb7 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:26.126354 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:26.155438 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34c665-5f97-4cba-a09e-32cd386840e7 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:26.156081 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34c665-5f97-4cba-a09e-32cd386840e7 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:26.156577 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34c665-5f97-4cba-a09e-32cd386840e7 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:26.158184 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 194/390] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:26.164049 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae33861-203d-4f1f-8e66-35a6dfb0758f None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:26.164577 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae33861-203d-4f1f-8e66-35a6dfb0758f None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:26.165098 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae33861-203d-4f1f-8e66-35a6dfb0758f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:26.167435 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a37ae-bb6f-472d-a816-77c34bb96dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:26.168246 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a37ae-bb6f-472d-a816-77c34bb96dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:26.168523 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a37ae-bb6f-472d-a816-77c34bb96dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:26.168792 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a37ae-bb6f-472d-a816-77c34bb96dd5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:26.184409 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-aae33861-203d-4f1f-8e66-35a6dfb0758f None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:26.186126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-aae33861-203d-4f1f-8e66-35a6dfb0758f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:26.188554 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0a37ae-bb6f-472d-a816-77c34bb96dd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:26.196484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-7c0a37ae-bb6f-472d-a816-77c34bb96dd5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:09:26.196484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:26.196484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:26.196484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:26.196484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:26.196484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:26.196484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:26.196484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:26.196484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:26.196484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:26.196484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:26.196484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:26.196484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:26.196484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:26.196484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:26.196484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:26.197690 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:26.197690 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:26.197690 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:26.197690 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:26.197690 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:26.197690 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:26.197690 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:26.197690 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:26.197690 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:26.197690 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:26.197690 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:26.197690 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:26.197690 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:26.197690 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:26.197690 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:26.197690 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:26.198688 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:26.198688 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:26.198688 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:26.198688 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:26.198688 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:26.198688 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:26.198688 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:26.198688 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:26.198688 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:26.198688 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:26.198688 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:09:26.198688 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:26.198688 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 197/391] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:09:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:26.204097 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a37ae-bb6f-472d-a816-77c34bb96dd5 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:26.323149 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 195/392] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:09:26.359110 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a37ae-bb6f-472d-a816-77c34bb96dd5 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:26.360996 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae33861-203d-4f1f-8e66-35a6dfb0758f None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:26.401397 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:26.405234 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647b15d-abb8-4583-9158-e31a2eeeeadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:26.405921 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647b15d-abb8-4583-9158-e31a2eeeeadb None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:26.406195 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647b15d-abb8-4583-9158-e31a2eeeeadb None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:26.406466 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b647b15d-abb8-4583-9158-e31a2eeeeadb None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:26.408026 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b647b15d-abb8-4583-9158-e31a2eeeeadb None admin] RBAC: Adding query filter params (name=member) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:26.408335 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b647b15d-abb8-4583-9158-e31a2eeeeadb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:26.409810 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b647b15d-abb8-4583-9158-e31a2eeeeadb None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:26.431271 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 198/393] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 03:09:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:26.442021 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647b15d-abb8-4583-9158-e31a2eeeeadb None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:26.495566 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae33861-203d-4f1f-8e66-35a6dfb0758f None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:26.523041 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:26.546991 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a0201-724e-4a9e-a402-b82fc2453c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:26.548128 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a0201-724e-4a9e-a402-b82fc2453c2a None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:26.548648 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a0201-724e-4a9e-a402-b82fc2453c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:26.549101 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a0201-724e-4a9e-a402-b82fc2453c2a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:26.560733 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a0201-724e-4a9e-a402-b82fc2453c2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:26.569133 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-a24a0201-724e-4a9e-a402-b82fc2453c2a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:09:26.569133 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:26.569133 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:26.569133 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:26.569133 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:26.569133 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:26.569133 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:26.569133 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:26.569133 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:26.569133 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:26.569133 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:26.569133 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:26.569133 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:26.569133 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:26.569133 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:26.569133 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:26.570478 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:26.570478 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:26.570478 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:26.570478 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:26.570478 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:26.570478 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:26.570478 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:26.570478 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:26.570478 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:26.570478 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:26.570478 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:26.570478 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:26.570478 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:26.570478 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:26.570478 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:26.570478 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:26.571272 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:26.571272 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:26.571272 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:26.571272 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:26.571272 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:26.571272 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:26.571272 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:26.571272 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:26.571272 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:26.571272 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:26.571272 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:09:26.571272 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:26.574219 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 196/394] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:09:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:26.580554 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a0201-724e-4a9e-a402-b82fc2453c2a None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:26.643245 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b647b15d-abb8-4583-9158-e31a2eeeeadb None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:26.686349 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af3bad-0665-409a-b7ad-089ae9431676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:26.687620 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af3bad-0665-409a-b7ad-089ae9431676 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:26.688098 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af3bad-0665-409a-b7ad-089ae9431676 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:26.688567 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af3bad-0665-409a-b7ad-089ae9431676 None admin] PATH_INFO: `/v3/groups/fa3d562d2df749e18edd0819748e3e90` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:26.690818 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85af3bad-0665-409a-b7ad-089ae9431676 None admin] RBAC: Authorizing `identity:get_group(group_id=fa3d562d2df749e18edd0819748e3e90)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:26.692669 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85af3bad-0665-409a-b7ad-089ae9431676 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:26.695370 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 199/395] 10.222.0.50 () {56 vars in 1180 bytes} [Fri May 27 03:09:26 2022] GET /identity/v3/groups/fa3d562d2df749e18edd0819748e3e90 => generated 226 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:26.701048 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af3bad-0665-409a-b7ad-089ae9431676 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:26.805332 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:26.806836 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:26.813319 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af3bad-0665-409a-b7ad-089ae9431676 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:26.838184 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a0201-724e-4a9e-a402-b82fc2453c2a None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:26.850011 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e384590e-918b-4cf8-9721-ef62c42bfefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:26.851731 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e384590e-918b-4cf8-9721-ef62c42bfefb None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:26.852186 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e384590e-918b-4cf8-9721-ef62c42bfefb None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:26.852731 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e384590e-918b-4cf8-9721-ef62c42bfefb None admin] PATH_INFO: `/v3/projects/4edef8e255584377bf94a9ac9712b4b7` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:26.854770 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e384590e-918b-4cf8-9721-ef62c42bfefb None admin] RBAC: Authorizing `identity:get_project(project_id=4edef8e255584377bf94a9ac9712b4b7)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:26.856886 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e384590e-918b-4cf8-9721-ef62c42bfefb None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:26.859080 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 200/396] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:09:26 2022] GET /identity/v3/projects/4edef8e255584377bf94a9ac9712b4b7 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:26.865259 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e384590e-918b-4cf8-9721-ef62c42bfefb None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:26.909802 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802f6b3-c7bd-492d-b41f-0d83fd3f5aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:26.910823 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802f6b3-c7bd-492d-b41f-0d83fd3f5aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:26.911100 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802f6b3-c7bd-492d-b41f-0d83fd3f5aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:26.911365 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802f6b3-c7bd-492d-b41f-0d83fd3f5aa8 None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:26.912544 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a802f6b3-c7bd-492d-b41f-0d83fd3f5aa8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:26.912862 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a802f6b3-c7bd-492d-b41f-0d83fd3f5aa8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:26.914356 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a802f6b3-c7bd-492d-b41f-0d83fd3f5aa8 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:26.939365 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 197/397] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 03:09:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:26.945523 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802f6b3-c7bd-492d-b41f-0d83fd3f5aa8 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:26.977407 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e384590e-918b-4cf8-9721-ef62c42bfefb None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:27.015669 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01cfbbc-3b81-4cdd-b61c-148c8709c50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:27.017019 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01cfbbc-3b81-4cdd-b61c-148c8709c50c None admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:27.017570 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01cfbbc-3b81-4cdd-b61c-148c8709c50c None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:27.018346 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01cfbbc-3b81-4cdd-b61c-148c8709c50c None admin] PATH_INFO: `/v3/projects/4edef8e255584377bf94a9ac9712b4b7/groups/fa3d562d2df749e18edd0819748e3e90/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:27.020813 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01cfbbc-3b81-4cdd-b61c-148c8709c50c None admin] RBAC: Authorizing `identity:create_grant(project_id=4edef8e255584377bf94a9ac9712b4b7, group_id=fa3d562d2df749e18edd0819748e3e90, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:27.022679 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01cfbbc-3b81-4cdd-b61c-148c8709c50c None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:27.041344 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 201/398] 10.222.0.50 () {58 vars in 1442 bytes} [Fri May 27 03:09:26 2022] PUT /identity/v3/projects/4edef8e255584377bf94a9ac9712b4b7/groups/fa3d562d2df749e18edd0819748e3e90/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:09:27.149799 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802f6b3-c7bd-492d-b41f-0d83fd3f5aa8 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:27.213841 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea323b7-53c3-4a7b-aa45-521d705d5d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:27.215083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea323b7-53c3-4a7b-aa45-521d705d5d1e None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:27.215566 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea323b7-53c3-4a7b-aa45-521d705d5d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:27.216035 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea323b7-53c3-4a7b-aa45-521d705d5d1e None admin] PATH_INFO: `/v3/groups/fa3d562d2df749e18edd0819748e3e90` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:27.217556 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea323b7-53c3-4a7b-aa45-521d705d5d1e None admin] RBAC: Authorizing `identity:get_group(group_id=fa3d562d2df749e18edd0819748e3e90)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:27.219219 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea323b7-53c3-4a7b-aa45-521d705d5d1e None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:27.221223 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 198/399] 10.222.0.50 () {56 vars in 1180 bytes} [Fri May 27 03:09:26 2022] GET /identity/v3/groups/fa3d562d2df749e18edd0819748e3e90 => generated 226 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:27.227078 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01cfbbc-3b81-4cdd-b61c-148c8709c50c None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:27.267855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:27.377112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01cfbbc-3b81-4cdd-b61c-148c8709c50c None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:27.413002 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f941e541-0d07-425e-9847-534e68b328b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:27.413880 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941e541-0d07-425e-9847-534e68b328b2 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:27.414154 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941e541-0d07-425e-9847-534e68b328b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:27.414421 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941e541-0d07-425e-9847-534e68b328b2 None admin] PATH_INFO: `/v3/projects/5e327e3411214877a9f427330ef40dc5` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:27.415617 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f941e541-0d07-425e-9847-534e68b328b2 None admin] RBAC: Authorizing `identity:get_project(project_id=5e327e3411214877a9f427330ef40dc5)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:27.417117 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f941e541-0d07-425e-9847-534e68b328b2 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:27.419135 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 202/400] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:09:27 2022] GET /identity/v3/projects/5e327e3411214877a9f427330ef40dc5 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:27.424737 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea323b7-53c3-4a7b-aa45-521d705d5d1e None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:27.428107 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:27.526075 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee9a08-4e57-4ac5-bfd9-cd548d01e7db None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:27.526247 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee9a08-4e57-4ac5-bfd9-cd548d01e7db None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:27.526531 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ee9a08-4e57-4ac5-bfd9-cd548d01e7db None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:27.528156 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 203/401] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:27.533244 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228b5f7-c81b-45c2-a871-0c60ed583949 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:27.533558 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228b5f7-c81b-45c2-a871-0c60ed583949 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:27.533785 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b228b5f7-c81b-45c2-a871-0c60ed583949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:27.553628 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-b228b5f7-c81b-45c2-a871-0c60ed583949 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:27.554827 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-b228b5f7-c81b-45c2-a871-0c60ed583949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:27.598027 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea323b7-53c3-4a7b-aa45-521d705d5d1e None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:27.634470 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd0ec7-b1a4-4a98-9c98-626596bdbbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:27.635373 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd0ec7-b1a4-4a98-9c98-626596bdbbf7 None admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:27.635650 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd0ec7-b1a4-4a98-9c98-626596bdbbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:27.635926 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd0ec7-b1a4-4a98-9c98-626596bdbbf7 None admin] PATH_INFO: `/v3/projects/5e327e3411214877a9f427330ef40dc5/groups/fa3d562d2df749e18edd0819748e3e90/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:27.637919 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cd0ec7-b1a4-4a98-9c98-626596bdbbf7 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e327e3411214877a9f427330ef40dc5, group_id=fa3d562d2df749e18edd0819748e3e90, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:27.639370 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cd0ec7-b1a4-4a98-9c98-626596bdbbf7 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:27.651282 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 204/402] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:27.653714 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5813568-00bf-425e-87c4-66efc119f7d4 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:27.654460 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5813568-00bf-425e-87c4-66efc119f7d4 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:27.654742 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5813568-00bf-425e-87c4-66efc119f7d4 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:27.657430 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 199/404] 10.222.0.50 () {58 vars in 1442 bytes} [Fri May 27 03:09:27 2022] PUT /identity/v3/projects/5e327e3411214877a9f427330ef40dc5/groups/fa3d562d2df749e18edd0819748e3e90/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:09:27.665002 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 205/404] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:27.673347 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da845a12-b645-4e28-8bfb-b59741642cb5 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:27.673559 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5813568-00bf-425e-87c4-66efc119f7d4 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:27.674147 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da845a12-b645-4e28-8bfb-b59741642cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:27.674632 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da845a12-b645-4e28-8bfb-b59741642cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:27.706474 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-da845a12-b645-4e28-8bfb-b59741642cb5 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:27.707694 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-da845a12-b645-4e28-8bfb-b59741642cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:27.725315 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:27.821326 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:27.849672 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 200/405] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:27.855226 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5813568-00bf-425e-87c4-66efc119f7d4 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:27.889142 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da845a12-b645-4e28-8bfb-b59741642cb5 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:27.891729 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe4cf4-1998-438a-b1ca-5c5d89860daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:27.892608 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe4cf4-1998-438a-b1ca-5c5d89860daf None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:27.892883 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe4cf4-1998-438a-b1ca-5c5d89860daf None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:27.893195 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe4cf4-1998-438a-b1ca-5c5d89860daf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:27.909404 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abe4cf4-1998-438a-b1ca-5c5d89860daf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:27.919803 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-5abe4cf4-1998-438a-b1ca-5c5d89860daf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:09:27.919803 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:27.919803 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:27.919803 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:27.919803 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:27.919803 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:27.919803 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:27.919803 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:27.919803 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:27.919803 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:27.919803 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:27.919803 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:27.919803 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:27.919803 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:27.919803 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:27.919803 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:27.920820 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:27.920820 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:27.920820 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:27.920820 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:27.920820 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:27.920820 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:27.920820 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:27.920820 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:27.920820 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:27.920820 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:27.920820 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:27.920820 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:27.920820 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:27.920820 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:27.920820 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:27.920820 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:27.921637 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:27.921637 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:27.921637 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:27.921637 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:27.921637 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:27.921637 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:27.921637 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:27.921637 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:27.921637 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:27.921637 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:27.921637 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:09:27.921637 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:27.926537 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 206/406] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 03:09:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:27.927908 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe4cf4-1998-438a-b1ca-5c5d89860daf None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:28.045283 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abe4cf4-1998-438a-b1ca-5c5d89860daf None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:28.049288 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:28.082745 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71cacd1-fb34-4f02-802d-168da365ecd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:28.083344 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71cacd1-fb34-4f02-802d-168da365ecd6 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:28.083758 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71cacd1-fb34-4f02-802d-168da365ecd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:28.084336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71cacd1-fb34-4f02-802d-168da365ecd6 None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:28.085288 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71cacd1-fb34-4f02-802d-168da365ecd6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:28.085592 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71cacd1-fb34-4f02-802d-168da365ecd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:28.087248 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71cacd1-fb34-4f02-802d-168da365ecd6 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:28.095117 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da845a12-b645-4e28-8bfb-b59741642cb5 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:28.099243 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 207/407] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:09:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:28.106155 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71cacd1-fb34-4f02-802d-168da365ecd6 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:28.214439 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71cacd1-fb34-4f02-802d-168da365ecd6 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:28.237701 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9853dc5-6363-4cef-ba20-ba36be00fbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:28.238688 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9853dc5-6363-4cef-ba20-ba36be00fbc3 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:28.239066 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9853dc5-6363-4cef-ba20-ba36be00fbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:28.239437 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9853dc5-6363-4cef-ba20-ba36be00fbc3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:28.248870 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8865db-8005-4aad-8a10-55b16bbdfcd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:28.249764 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8865db-8005-4aad-8a10-55b16bbdfcd7 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:28.250038 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8865db-8005-4aad-8a10-55b16bbdfcd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:28.250306 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8865db-8005-4aad-8a10-55b16bbdfcd7 None admin] PATH_INFO: `/v3/groups/fa3d562d2df749e18edd0819748e3e90` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:28.251631 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8865db-8005-4aad-8a10-55b16bbdfcd7 None admin] RBAC: Authorizing `identity:get_group(group_id=fa3d562d2df749e18edd0819748e3e90)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:28.253251 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8865db-8005-4aad-8a10-55b16bbdfcd7 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:28.255598 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 208/408] 10.222.0.50 () {56 vars in 1180 bytes} [Fri May 27 03:09:28 2022] GET /identity/v3/groups/fa3d562d2df749e18edd0819748e3e90 => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:28.261194 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8865db-8005-4aad-8a10-55b16bbdfcd7 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:28.274197 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9853dc5-6363-4cef-ba20-ba36be00fbc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:28.290180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-b9853dc5-6363-4cef-ba20-ba36be00fbc3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:09:28.290180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:28.290180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:28.290180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:28.290180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:28.290180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:28.290180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:28.290180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:28.290180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:28.290180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:28.290180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:28.290180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:28.290180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:28.290180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:28.290180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:28.290180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:28.291078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:28.291078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:28.291078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:28.291078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:28.291078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:28.291078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:28.291078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:28.291078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:28.291078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:28.291078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:28.291078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:28.291078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:28.291078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:28.291078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:28.291078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:28.291078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:28.291839 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:28.291839 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:28.291839 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:28.291839 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:28.291839 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:28.291839 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:28.291839 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:28.291839 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:28.291839 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:28.291839 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:28.291839 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:09:28.291839 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:28.291839 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 201/409] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 03:09:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 403 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:09:28.297316 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:28.303772 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9853dc5-6363-4cef-ba20-ba36be00fbc3 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:28.357437 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:28.382667 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8865db-8005-4aad-8a10-55b16bbdfcd7 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:28.408366 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9853dc5-6363-4cef-ba20-ba36be00fbc3 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:28.426703 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15598f9-b4d0-4a2b-8e4c-6ad306f648c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:28.427897 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15598f9-b4d0-4a2b-8e4c-6ad306f648c0 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:28.428177 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15598f9-b4d0-4a2b-8e4c-6ad306f648c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:28.428874 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15598f9-b4d0-4a2b-8e4c-6ad306f648c0 None admin] PATH_INFO: `/v3/projects/5e327e3411214877a9f427330ef40dc5` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:28.430373 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15598f9-b4d0-4a2b-8e4c-6ad306f648c0 None admin] RBAC: Authorizing `identity:get_project(project_id=5e327e3411214877a9f427330ef40dc5)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:28.432293 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15598f9-b4d0-4a2b-8e4c-6ad306f648c0 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:28.437042 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 209/410] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:09:28 2022] GET /identity/v3/projects/5e327e3411214877a9f427330ef40dc5 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:28.438372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad3737e-aca6-40ca-8400-264a8f40af03 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:28.438655 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad3737e-aca6-40ca-8400-264a8f40af03 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:28.438928 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad3737e-aca6-40ca-8400-264a8f40af03 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:28.440414 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 210/411] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:28.441348 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad3737e-aca6-40ca-8400-264a8f40af03 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:28.456974 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9340d020-6ace-4910-a653-c1cbac094f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:28.456974 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340d020-6ace-4910-a653-c1cbac094f76 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:28.457227 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340d020-6ace-4910-a653-c1cbac094f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:28.457486 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340d020-6ace-4910-a653-c1cbac094f76 None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:28.458564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9340d020-6ace-4910-a653-c1cbac094f76 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:28.459230 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9340d020-6ace-4910-a653-c1cbac094f76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:28.461228 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9340d020-6ace-4910-a653-c1cbac094f76 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:28.473077 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 202/412] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:09:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:28.475116 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d92ae-0bb3-4fc1-af86-053a660b1586 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:28.475824 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d92ae-0bb3-4fc1-af86-053a660b1586 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:28.476102 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d92ae-0bb3-4fc1-af86-053a660b1586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:28.500609 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-ba7d92ae-0bb3-4fc1-af86-053a660b1586 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:28.502358 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-ba7d92ae-0bb3-4fc1-af86-053a660b1586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:28.538299 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:28.564203 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad3737e-aca6-40ca-8400-264a8f40af03 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:28.589113 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 203/413] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:28.597126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7d92ae-0bb3-4fc1-af86-053a660b1586 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:28.620068 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cfccaf-1056-446b-aa62-631a9daaf4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:28.621796 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfccaf-1056-446b-aa62-631a9daaf4a5 None admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:28.622296 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfccaf-1056-446b-aa62-631a9daaf4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:28.622809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfccaf-1056-446b-aa62-631a9daaf4a5 None admin] PATH_INFO: `/v3/projects/5e327e3411214877a9f427330ef40dc5/groups/fa3d562d2df749e18edd0819748e3e90/roles/9e63b73b3ce24bb3ad214ecdb6132696` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:28.626520 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cfccaf-1056-446b-aa62-631a9daaf4a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e327e3411214877a9f427330ef40dc5, group_id=fa3d562d2df749e18edd0819748e3e90, role_id=9e63b73b3ce24bb3ad214ecdb6132696)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:28.629057 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cfccaf-1056-446b-aa62-631a9daaf4a5 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:28.656389 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:28.664224 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 211/414] 10.222.0.50 () {58 vars in 1442 bytes} [Fri May 27 03:09:28 2022] PUT /identity/v3/projects/5e327e3411214877a9f427330ef40dc5/groups/fa3d562d2df749e18edd0819748e3e90/roles/9e63b73b3ce24bb3ad214ecdb6132696 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:09:28.664854 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cfccaf-1056-446b-aa62-631a9daaf4a5 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:28.735114 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7d92ae-0bb3-4fc1-af86-053a660b1586 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:28.776663 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a865b74-2591-4cac-9c6b-2189d4e847d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:28.778401 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a865b74-2591-4cac-9c6b-2189d4e847d7 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:28.779060 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a865b74-2591-4cac-9c6b-2189d4e847d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:28.779529 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a865b74-2591-4cac-9c6b-2189d4e847d7 None admin] PATH_INFO: `/v3/groups/e78521d2f9ed4d93ad55d88822834e22` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:28.780987 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a865b74-2591-4cac-9c6b-2189d4e847d7 None admin] RBAC: Authorizing `identity:get_group(group_id=e78521d2f9ed4d93ad55d88822834e22)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:28.782856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a865b74-2591-4cac-9c6b-2189d4e847d7 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:28.785480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 204/415] 10.222.0.50 () {56 vars in 1180 bytes} [Fri May 27 03:09:28 2022] GET /identity/v3/groups/e78521d2f9ed4d93ad55d88822834e22 => generated 229 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:28.793466 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a865b74-2591-4cac-9c6b-2189d4e847d7 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:28.918702 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a865b74-2591-4cac-9c6b-2189d4e847d7 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:28.935556 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:28.958211 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cfccaf-1056-446b-aa62-631a9daaf4a5 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:28.970767 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fea181-c231-457b-8324-b424731ce186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:28.970767 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fea181-c231-457b-8324-b424731ce186 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:28.971047 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fea181-c231-457b-8324-b424731ce186 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:28.971291 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fea181-c231-457b-8324-b424731ce186 None admin] PATH_INFO: `/v3/projects/73644351894d46fdb8889fca96b4c614` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:28.972713 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fea181-c231-457b-8324-b424731ce186 None admin] RBAC: Authorizing `identity:get_project(project_id=73644351894d46fdb8889fca96b4c614)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:28.973964 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fea181-c231-457b-8324-b424731ce186 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:28.976247 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 205/416] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:09:28 2022] GET /identity/v3/projects/73644351894d46fdb8889fca96b4c614 => generated 344 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:28.981614 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fea181-c231-457b-8324-b424731ce186 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:28.996906 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c157b3d8-4d44-4bcc-97a0-2ce1331cff97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:28.997812 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157b3d8-4d44-4bcc-97a0-2ce1331cff97 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:28.998013 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157b3d8-4d44-4bcc-97a0-2ce1331cff97 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:28.998282 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157b3d8-4d44-4bcc-97a0-2ce1331cff97 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:29.007165 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:29.013432 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c157b3d8-4d44-4bcc-97a0-2ce1331cff97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:29.025782 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-c157b3d8-4d44-4bcc-97a0-2ce1331cff97 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:09:29.025782 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:29.025782 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:29.025782 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:29.025782 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:29.025782 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:29.025782 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:29.025782 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:29.025782 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:29.025782 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:29.025782 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:29.025782 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:29.025782 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:29.025782 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:29.025782 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:29.025782 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:29.026750 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:29.026750 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:29.026750 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:29.026750 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:29.026750 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:29.026750 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:29.026750 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:29.026750 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:29.026750 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:29.026750 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:29.026750 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:29.026750 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:29.026750 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:29.026750 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:29.026750 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:29.026750 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:29.027580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:29.027580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:29.027580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:29.027580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:29.027580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:29.027580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:29.027580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:29.027580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:29.027580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:29.027580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:29.027580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:09:29.027580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:29.033228 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 212/417] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 03:09:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:29.035358 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c157b3d8-4d44-4bcc-97a0-2ce1331cff97 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:29.132901 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fea181-c231-457b-8324-b424731ce186 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:29.198639 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad59ccfe-82a4-44d1-adcf-66f87980fcf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:29.199539 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad59ccfe-82a4-44d1-adcf-66f87980fcf9 None admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:29.199861 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad59ccfe-82a4-44d1-adcf-66f87980fcf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:29.200133 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad59ccfe-82a4-44d1-adcf-66f87980fcf9 None admin] PATH_INFO: `/v3/projects/73644351894d46fdb8889fca96b4c614/groups/e78521d2f9ed4d93ad55d88822834e22/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:29.203259 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad59ccfe-82a4-44d1-adcf-66f87980fcf9 None admin] RBAC: Authorizing `identity:create_grant(project_id=73644351894d46fdb8889fca96b4c614, group_id=e78521d2f9ed4d93ad55d88822834e22, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:29.204945 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad59ccfe-82a4-44d1-adcf-66f87980fcf9 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:29.223011 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c157b3d8-4d44-4bcc-97a0-2ce1331cff97 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:29.229439 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 206/418] 10.222.0.50 () {58 vars in 1442 bytes} [Fri May 27 03:09:28 2022] PUT /identity/v3/projects/73644351894d46fdb8889fca96b4c614/groups/e78521d2f9ed4d93ad55d88822834e22/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:09:29.282991 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f7b5fc-431f-4e26-9249-363f5e0452f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:29.283937 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7b5fc-431f-4e26-9249-363f5e0452f6 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:29.284216 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7b5fc-431f-4e26-9249-363f5e0452f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:29.284491 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7b5fc-431f-4e26-9249-363f5e0452f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:29.285710 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f7b5fc-431f-4e26-9249-363f5e0452f6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:29.286069 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f7b5fc-431f-4e26-9249-363f5e0452f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:29.288010 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f7b5fc-431f-4e26-9249-363f5e0452f6 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:29.301797 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 213/419] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:09:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:29.307225 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad59ccfe-82a4-44d1-adcf-66f87980fcf9 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:29.360068 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:29.435144 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad59ccfe-82a4-44d1-adcf-66f87980fcf9 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:29.476479 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc392f9-87cb-4fd0-b7a8-582392ec25b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:29.477117 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc392f9-87cb-4fd0-b7a8-582392ec25b5 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:29.477403 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc392f9-87cb-4fd0-b7a8-582392ec25b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:29.477680 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc392f9-87cb-4fd0-b7a8-582392ec25b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:29.554873 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc392f9-87cb-4fd0-b7a8-582392ec25b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:29.556383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc392f9-87cb-4fd0-b7a8-582392ec25b5 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:29.599483 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:29.661113 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 207/420] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:09:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:09:29.670100 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f7b5fc-431f-4e26-9249-363f5e0452f6 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:29.765133 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:29.816291 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f7b5fc-431f-4e26-9249-363f5e0452f6 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:29.843659 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bea37f-3a32-41a5-8beb-806e9591a8c4 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:29.843946 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bea37f-3a32-41a5-8beb-806e9591a8c4 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:29.844256 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bea37f-3a32-41a5-8beb-806e9591a8c4 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:29.845815 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 208/421] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:09:29.852815 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40606ca-cda7-4490-9e24-407bf3270836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:29.853519 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a99bbd-7c26-4ae3-8211-7095f67012f7 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:29.853803 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a99bbd-7c26-4ae3-8211-7095f67012f7 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:29.854080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a99bbd-7c26-4ae3-8211-7095f67012f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:29.854618 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40606ca-cda7-4490-9e24-407bf3270836 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:29.855081 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40606ca-cda7-4490-9e24-407bf3270836 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:29.855556 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40606ca-cda7-4490-9e24-407bf3270836 None admin] PATH_INFO: `/v3/projects/4edef8e255584377bf94a9ac9712b4b7` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:29.856939 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40606ca-cda7-4490-9e24-407bf3270836 None admin] RBAC: Authorizing `identity:get_project(project_id=4edef8e255584377bf94a9ac9712b4b7)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:29.858605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40606ca-cda7-4490-9e24-407bf3270836 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:29.860777 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 214/422] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:09:29 2022] GET /identity/v3/projects/4edef8e255584377bf94a9ac9712b4b7 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:29.866141 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40606ca-cda7-4490-9e24-407bf3270836 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:29.886028 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-73a99bbd-7c26-4ae3-8211-7095f67012f7 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:29.887253 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-73a99bbd-7c26-4ae3-8211-7095f67012f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:30.019967 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 209/423] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:30.023065 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40606ca-cda7-4490-9e24-407bf3270836 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:30.058048 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:30.061935 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a99bbd-7c26-4ae3-8211-7095f67012f7 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:30.079482 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8175b211-524e-44a1-a7c2-9e5bfe73d985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:30.081948 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175b211-524e-44a1-a7c2-9e5bfe73d985 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:30.084416 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175b211-524e-44a1-a7c2-9e5bfe73d985 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:30.084935 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175b211-524e-44a1-a7c2-9e5bfe73d985 None admin] PATH_INFO: `/v3/groups/fa3d562d2df749e18edd0819748e3e90` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:30.087315 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8175b211-524e-44a1-a7c2-9e5bfe73d985 None admin] RBAC: Authorizing `identity:get_group(group_id=fa3d562d2df749e18edd0819748e3e90)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:30.090341 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8175b211-524e-44a1-a7c2-9e5bfe73d985 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:30.097362 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 215/424] 10.222.0.50 () {56 vars in 1180 bytes} [Fri May 27 03:09:29 2022] GET /identity/v3/groups/fa3d562d2df749e18edd0819748e3e90 => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:30.099166 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8175b211-524e-44a1-a7c2-9e5bfe73d985 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:30.199254 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:30.233780 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a99bbd-7c26-4ae3-8211-7095f67012f7 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:30.271448 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a752a80-79ff-48ca-a43c-d2f6ac518264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:30.272704 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a752a80-79ff-48ca-a43c-d2f6ac518264 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:30.272932 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8175b211-524e-44a1-a7c2-9e5bfe73d985 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:30.273424 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a752a80-79ff-48ca-a43c-d2f6ac518264 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:30.273889 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a752a80-79ff-48ca-a43c-d2f6ac518264 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:30.285421 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a752a80-79ff-48ca-a43c-d2f6ac518264 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:30.295490 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-4a752a80-79ff-48ca-a43c-d2f6ac518264 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:09:30.295490 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:30.295490 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:30.295490 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:30.295490 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:30.295490 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:30.295490 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:30.295490 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:30.295490 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:30.295490 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:30.295490 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:30.295490 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:30.295490 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:30.295490 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:30.295490 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:30.295490 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:30.296448 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:30.296448 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:30.296448 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:30.296448 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:30.296448 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:30.296448 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:30.296448 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:30.296448 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:30.296448 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:30.296448 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:30.296448 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:30.296448 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:30.296448 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:30.296448 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:30.296448 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:30.296448 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:30.297242 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:30.297242 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:30.297242 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:30.297242 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:30.297242 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:30.297242 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:30.297242 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:30.297242 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:30.297242 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:30.297242 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:30.297242 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:09:30.297242 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:30.300109 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 210/425] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:09:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:30.306335 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a752a80-79ff-48ca-a43c-d2f6ac518264 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:30.349934 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233c9e7c-235a-4ef2-8673-03ef27afa08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:30.350771 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c9e7c-235a-4ef2-8673-03ef27afa08a None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:30.350904 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c9e7c-235a-4ef2-8673-03ef27afa08a None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:30.351146 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c9e7c-235a-4ef2-8673-03ef27afa08a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:30.351979 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233c9e7c-235a-4ef2-8673-03ef27afa08a None admin] RBAC: Adding query filter params (group.id=fa3d562d2df749e18edd0819748e3e90, role.id=9e63b73b3ce24bb3ad214ecdb6132696, scope.project.id=4edef8e255584377bf94a9ac9712b4b7) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:30.352288 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233c9e7c-235a-4ef2-8673-03ef27afa08a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:30.353817 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233c9e7c-235a-4ef2-8673-03ef27afa08a None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:30.369439 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 216/426] 10.222.0.50 () {56 vars in 1376 bytes} [Fri May 27 03:09:30 2022] GET /identity/v3/role_assignments?group.id=fa3d562d2df749e18edd0819748e3e90&scope.project.id=4edef8e255584377bf94a9ac9712b4b7&role.id=9e63b73b3ce24bb3ad214ecdb6132696 => generated 611 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:30.373810 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3a6a0-a6f7-4c77-a2c9-11dd88d71db2 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:30.374084 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3a6a0-a6f7-4c77-a2c9-11dd88d71db2 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:30.374358 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3a6a0-a6f7-4c77-a2c9-11dd88d71db2 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:30.375932 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 217/427] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:30.381812 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc032c-6d87-49e5-a2aa-eca794100ff1 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:30.381812 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc032c-6d87-49e5-a2aa-eca794100ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:30.382076 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfc032c-6d87-49e5-a2aa-eca794100ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:30.407417 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-edfc032c-6d87-49e5-a2aa-eca794100ff1 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:30.408525 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-edfc032c-6d87-49e5-a2aa-eca794100ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:30.419691 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a752a80-79ff-48ca-a43c-d2f6ac518264 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:30.452913 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:30.460513 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e434825-41db-444d-bbe5-7c3615aba7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:30.461629 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e434825-41db-444d-bbe5-7c3615aba7c1 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:30.462247 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e434825-41db-444d-bbe5-7c3615aba7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:30.462797 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e434825-41db-444d-bbe5-7c3615aba7c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:30.463779 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e434825-41db-444d-bbe5-7c3615aba7c1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:30.464291 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e434825-41db-444d-bbe5-7c3615aba7c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:30.466007 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e434825-41db-444d-bbe5-7c3615aba7c1 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:30.477482 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 211/428] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 03:09:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:30.483015 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e434825-41db-444d-bbe5-7c3615aba7c1 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:30.552557 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:30.583551 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 218/429] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:30.604072 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e434825-41db-444d-bbe5-7c3615aba7c1 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:30.620769 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfc032c-6d87-49e5-a2aa-eca794100ff1 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:30.642501 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53eab8d-5199-4d53-bb75-b9bcbe65f6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:30.643564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53eab8d-5199-4d53-bb75-b9bcbe65f6b2 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:30.643988 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53eab8d-5199-4d53-bb75-b9bcbe65f6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:30.644400 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53eab8d-5199-4d53-bb75-b9bcbe65f6b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:30.711152 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:30.724782 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53eab8d-5199-4d53-bb75-b9bcbe65f6b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:30.726689 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53eab8d-5199-4d53-bb75-b9bcbe65f6b2 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:30.731501 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfc032c-6d87-49e5-a2aa-eca794100ff1 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:30.769951 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca353a-2a8b-4499-88c7-58d99de0e95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:30.770889 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca353a-2a8b-4499-88c7-58d99de0e95f None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:30.771344 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca353a-2a8b-4499-88c7-58d99de0e95f None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:30.771681 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca353a-2a8b-4499-88c7-58d99de0e95f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:30.782405 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ca353a-2a8b-4499-88c7-58d99de0e95f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:30.791422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-31ca353a-2a8b-4499-88c7-58d99de0e95f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 03:09:30.791422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:30.791422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:30.791422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:30.791422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:30.791422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:30.791422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:30.791422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:30.791422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:30.791422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:30.791422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:30.791422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:30.791422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:30.791422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:30.791422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:30.791422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:30.792352 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:30.792352 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:30.792352 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:30.792352 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:30.792352 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:30.792352 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:30.792352 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:30.792352 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:30.792352 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:30.792352 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:30.792352 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:30.792352 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:30.792352 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:30.792352 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:30.792352 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:30.792352 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:30.793168 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:30.793168 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:30.793168 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:30.793168 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:30.793168 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:30.793168 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:30.793168 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:30.793168 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:30.793168 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:30.793168 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:30.793168 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 03:09:30.793168 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:30.793168 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 219/430] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:09:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:09:30.798780 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca353a-2a8b-4499-88c7-58d99de0e95f None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:30.845229 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 212/431] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:09:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:09:30.851985 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53eab8d-5199-4d53-bb75-b9bcbe65f6b2 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:30.861385 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:30.902687 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca353a-2a8b-4499-88c7-58d99de0e95f None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:30.937070 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4603940a-e2cf-4a9e-a7e4-d280be7e8e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:30.937749 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4603940a-e2cf-4a9e-a7e4-d280be7e8e5c None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:30.938030 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4603940a-e2cf-4a9e-a7e4-d280be7e8e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:30.938308 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4603940a-e2cf-4a9e-a7e4-d280be7e8e5c None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:30.939052 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4603940a-e2cf-4a9e-a7e4-d280be7e8e5c None admin] RBAC: Adding query filter params (name=member) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:30.939355 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4603940a-e2cf-4a9e-a7e4-d280be7e8e5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:30.940763 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4603940a-e2cf-4a9e-a7e4-d280be7e8e5c None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:30.949343 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 220/432] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 03:09:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:30.953468 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:30.954834 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4603940a-e2cf-4a9e-a7e4-d280be7e8e5c None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:30.963466 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53eab8d-5199-4d53-bb75-b9bcbe65f6b2 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:31.000901 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca7777-8578-4374-ae01-0542d7bdbbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:31.001869 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca7777-8578-4374-ae01-0542d7bdbbf9 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:31.002155 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca7777-8578-4374-ae01-0542d7bdbbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:31.002433 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca7777-8578-4374-ae01-0542d7bdbbf9 None admin] PATH_INFO: `/v3/projects/4edef8e255584377bf94a9ac9712b4b7` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:31.003624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ca7777-8578-4374-ae01-0542d7bdbbf9 None admin] RBAC: Authorizing `identity:get_project(project_id=4edef8e255584377bf94a9ac9712b4b7)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:31.005125 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ca7777-8578-4374-ae01-0542d7bdbbf9 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:31.007192 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 213/433] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:09:30 2022] GET /identity/v3/projects/4edef8e255584377bf94a9ac9712b4b7 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:31.012354 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca7777-8578-4374-ae01-0542d7bdbbf9 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:31.069830 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4603940a-e2cf-4a9e-a7e4-d280be7e8e5c None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:31.086073 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:31.106830 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba84c86e-41ce-4fb2-afaf-504516cc47da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:31.107707 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba84c86e-41ce-4fb2-afaf-504516cc47da None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:31.108127 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba84c86e-41ce-4fb2-afaf-504516cc47da None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:31.108493 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba84c86e-41ce-4fb2-afaf-504516cc47da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:31.142824 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca7777-8578-4374-ae01-0542d7bdbbf9 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:31.179721 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632da7bb-10a7-4e3e-af84-183c6bc4b003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:31.180863 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632da7bb-10a7-4e3e-af84-183c6bc4b003 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:31.181313 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632da7bb-10a7-4e3e-af84-183c6bc4b003 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:31.181728 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632da7bb-10a7-4e3e-af84-183c6bc4b003 None admin] PATH_INFO: `/v3/groups/fa3d562d2df749e18edd0819748e3e90` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:31.183167 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632da7bb-10a7-4e3e-af84-183c6bc4b003 None admin] RBAC: Authorizing `identity:get_group(group_id=fa3d562d2df749e18edd0819748e3e90)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:31.184857 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632da7bb-10a7-4e3e-af84-183c6bc4b003 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:31.191472 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba84c86e-41ce-4fb2-afaf-504516cc47da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:31.191472 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba84c86e-41ce-4fb2-afaf-504516cc47da None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:31.192496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 214/434] 10.222.0.50 () {56 vars in 1180 bytes} [Fri May 27 03:09:31 2022] GET /identity/v3/groups/fa3d562d2df749e18edd0819748e3e90 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:31.194555 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632da7bb-10a7-4e3e-af84-183c6bc4b003 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:31.229474 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:31.317060 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 221/435] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:09:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:09:31.321238 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba84c86e-41ce-4fb2-afaf-504516cc47da None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:31.331430 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632da7bb-10a7-4e3e-af84-183c6bc4b003 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:31.365560 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:31.375124 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454240d2-b2a6-47d3-8651-97390f276db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:31.375930 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454240d2-b2a6-47d3-8651-97390f276db7 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:31.376216 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454240d2-b2a6-47d3-8651-97390f276db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:31.376487 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454240d2-b2a6-47d3-8651-97390f276db7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:31.377637 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454240d2-b2a6-47d3-8651-97390f276db7 None admin] RBAC: Adding query filter params (group.id=fa3d562d2df749e18edd0819748e3e90, role.id=0129dd45ff5449f9957e8efdd01a55ed, scope.project.id=4edef8e255584377bf94a9ac9712b4b7) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:31.377949 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454240d2-b2a6-47d3-8651-97390f276db7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:31.379384 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454240d2-b2a6-47d3-8651-97390f276db7 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:31.387326 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 215/436] 10.222.0.50 () {56 vars in 1376 bytes} [Fri May 27 03:09:31 2022] GET /identity/v3/role_assignments?group.id=fa3d562d2df749e18edd0819748e3e90&scope.project.id=4edef8e255584377bf94a9ac9712b4b7&role.id=0129dd45ff5449f9957e8efdd01a55ed => generated 611 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:31.388586 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606797b6-4765-4294-872f-4304db88eff7 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:31.388866 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606797b6-4765-4294-872f-4304db88eff7 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:31.389161 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606797b6-4765-4294-872f-4304db88eff7 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:31.390504 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 216/437] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:31.396050 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35027b2b-9016-47c7-9bc7-bd87c223c6e6 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:31.396340 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35027b2b-9016-47c7-9bc7-bd87c223c6e6 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:31.396635 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35027b2b-9016-47c7-9bc7-bd87c223c6e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:31.422607 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-35027b2b-9016-47c7-9bc7-bd87c223c6e6 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:31.423928 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-35027b2b-9016-47c7-9bc7-bd87c223c6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:31.454887 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba84c86e-41ce-4fb2-afaf-504516cc47da None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:31.495347 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93feb350-f2b7-44ad-b591-824aba25afd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:31.496615 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93feb350-f2b7-44ad-b591-824aba25afd4 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:31.497189 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93feb350-f2b7-44ad-b591-824aba25afd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:31.497661 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93feb350-f2b7-44ad-b591-824aba25afd4 None admin] PATH_INFO: `/v3/projects/5e327e3411214877a9f427330ef40dc5` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:31.499222 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93feb350-f2b7-44ad-b591-824aba25afd4 None admin] RBAC: Authorizing `identity:get_project(project_id=5e327e3411214877a9f427330ef40dc5)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:31.501102 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93feb350-f2b7-44ad-b591-824aba25afd4 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:31.503607 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 222/438] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:09:31 2022] GET /identity/v3/projects/5e327e3411214877a9f427330ef40dc5 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:31.517117 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 217/439] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:09:31.518291 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93feb350-f2b7-44ad-b591-824aba25afd4 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:31.544770 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35027b2b-9016-47c7-9bc7-bd87c223c6e6 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:31.573577 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:31.631657 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:31.652845 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93feb350-f2b7-44ad-b591-824aba25afd4 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:31.676615 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35027b2b-9016-47c7-9bc7-bd87c223c6e6 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:31.696521 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f71b363-1b80-4aa1-b2f2-895a11378374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:31.697706 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f71b363-1b80-4aa1-b2f2-895a11378374 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:31.698101 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f71b363-1b80-4aa1-b2f2-895a11378374 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:31.698388 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f71b363-1b80-4aa1-b2f2-895a11378374 None admin] PATH_INFO: `/v3/groups/fa3d562d2df749e18edd0819748e3e90` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:31.699619 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f71b363-1b80-4aa1-b2f2-895a11378374 None admin] RBAC: Authorizing `identity:get_group(group_id=fa3d562d2df749e18edd0819748e3e90)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:31.701151 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f71b363-1b80-4aa1-b2f2-895a11378374 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:31.705040 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 223/440] 10.222.0.50 () {56 vars in 1180 bytes} [Fri May 27 03:09:31 2022] GET /identity/v3/groups/fa3d562d2df749e18edd0819748e3e90 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:31.708863 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f71b363-1b80-4aa1-b2f2-895a11378374 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:31.723082 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b4066-d82a-4648-aec4-a9af6b4f6c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:31.723937 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b4066-d82a-4648-aec4-a9af6b4f6c34 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:31.724226 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b4066-d82a-4648-aec4-a9af6b4f6c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:31.724476 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b4066-d82a-4648-aec4-a9af6b4f6c34 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:31.740815 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b4066-d82a-4648-aec4-a9af6b4f6c34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:31.751610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-fc3b4066-d82a-4648-aec4-a9af6b4f6c34 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:09:31.751610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:31.751610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:31.751610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:31.751610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:31.751610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:31.751610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:31.751610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:31.751610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:31.751610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:31.751610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:31.751610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:31.751610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:31.751610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:31.751610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:31.751610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:31.752522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:31.752522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:31.752522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:31.752522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:31.752522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:31.752522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:31.752522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:31.752522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:31.752522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:31.752522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:31.752522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:31.752522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:31.752522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:31.752522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:31.752522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:31.752522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:31.754218 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:31.754218 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:31.754218 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:31.754218 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:31.754218 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:31.754218 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:31.754218 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:31.754218 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:31.754218 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:31.754218 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:31.754218 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:09:31.754218 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:31.754218 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 218/441] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 03:09:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:09:31.759951 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b4066-d82a-4648-aec4-a9af6b4f6c34 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:31.838778 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f71b363-1b80-4aa1-b2f2-895a11378374 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:31.874098 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:31.884867 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f8207b-036d-45e1-8396-0be410ad0354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:31.885498 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8207b-036d-45e1-8396-0be410ad0354 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:31.885776 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8207b-036d-45e1-8396-0be410ad0354 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:31.886048 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8207b-036d-45e1-8396-0be410ad0354 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:31.886884 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f8207b-036d-45e1-8396-0be410ad0354 None admin] RBAC: Adding query filter params (group.id=fa3d562d2df749e18edd0819748e3e90, role.id=0129dd45ff5449f9957e8efdd01a55ed, scope.project.id=5e327e3411214877a9f427330ef40dc5) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:31.887196 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f8207b-036d-45e1-8396-0be410ad0354 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:31.888679 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f8207b-036d-45e1-8396-0be410ad0354 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:31.896187 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 224/442] 10.222.0.50 () {56 vars in 1376 bytes} [Fri May 27 03:09:31 2022] GET /identity/v3/role_assignments?group.id=fa3d562d2df749e18edd0819748e3e90&scope.project.id=5e327e3411214877a9f427330ef40dc5&role.id=0129dd45ff5449f9957e8efdd01a55ed => generated 611 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:31.897644 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3a9cf-542e-4e45-baa2-ff6f4de93ebf None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:31.898951 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b4066-d82a-4648-aec4-a9af6b4f6c34 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:31.899103 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3a9cf-542e-4e45-baa2-ff6f4de93ebf None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:31.899284 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3a9cf-542e-4e45-baa2-ff6f4de93ebf None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:31.901111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 225/443] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:31.910879 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db1874-a06e-47c8-a766-35389d024c85 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:31.911231 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db1874-a06e-47c8-a766-35389d024c85 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:31.912177 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db1874-a06e-47c8-a766-35389d024c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:31.931455 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-01db1874-a06e-47c8-a766-35389d024c85 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:31.932697 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-01db1874-a06e-47c8-a766-35389d024c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:31.944311 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19375533-68cf-4c39-b234-77965fe98774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:31.945180 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:31.945800 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19375533-68cf-4c39-b234-77965fe98774 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:31.946456 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19375533-68cf-4c39-b234-77965fe98774 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:31.947154 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19375533-68cf-4c39-b234-77965fe98774 None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:31.948764 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19375533-68cf-4c39-b234-77965fe98774 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:31.949399 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19375533-68cf-4c39-b234-77965fe98774 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:31.951713 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19375533-68cf-4c39-b234-77965fe98774 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:31.965536 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 219/444] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:09:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:31.967385 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19375533-68cf-4c39-b234-77965fe98774 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:32.026663 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 226/445] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:32.049987 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01db1874-a06e-47c8-a766-35389d024c85 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:32.104020 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19375533-68cf-4c39-b234-77965fe98774 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:32.153331 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad2ae8b-5010-4b2e-8574-41daccf6e7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:32.153921 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2ae8b-5010-4b2e-8574-41daccf6e7fa None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:32.154199 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2ae8b-5010-4b2e-8574-41daccf6e7fa None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:32.154468 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2ae8b-5010-4b2e-8574-41daccf6e7fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:32.172553 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:32.179913 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01db1874-a06e-47c8-a766-35389d024c85 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:32.219442 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830175f8-3752-4b3b-807e-4aecf7346157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:32.220644 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830175f8-3752-4b3b-807e-4aecf7346157 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:32.221142 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830175f8-3752-4b3b-807e-4aecf7346157 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:32.221417 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830175f8-3752-4b3b-807e-4aecf7346157 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:32.233374 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830175f8-3752-4b3b-807e-4aecf7346157 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:32.234985 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad2ae8b-5010-4b2e-8574-41daccf6e7fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:32.236487 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad2ae8b-5010-4b2e-8574-41daccf6e7fa None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:32.241318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-830175f8-3752-4b3b-807e-4aecf7346157 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:09:32.241318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:32.241318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:32.241318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:32.241318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:32.241318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:32.241318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:32.241318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:32.241318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:32.241318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:32.241318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:32.241318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:32.241318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:32.241318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:32.241318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:32.241318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:32.242462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:32.242462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:32.242462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:32.242462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:32.242462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:32.242462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:32.242462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:32.242462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:32.242462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:32.242462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:32.242462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:32.242462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:32.242462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:32.242462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:32.242462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:32.242462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:32.243422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:32.243422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:32.243422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:32.243422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:32.243422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:32.243422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:32.243422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:32.243422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:32.243422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:32.243422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:32.243422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:09:32.243422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:32.244106 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 227/446] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 03:09:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:09:32.250517 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830175f8-3752-4b3b-807e-4aecf7346157 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:32.255815 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:32.353903 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 220/447] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:09:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:09:32.359496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad2ae8b-5010-4b2e-8574-41daccf6e7fa None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:32.374291 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830175f8-3752-4b3b-807e-4aecf7346157 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:32.403889 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:32.407740 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7311b2b-c10a-4406-b8ae-b58a99bac70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:32.408403 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7311b2b-c10a-4406-b8ae-b58a99bac70c None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:32.408685 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7311b2b-c10a-4406-b8ae-b58a99bac70c None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:32.409017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7311b2b-c10a-4406-b8ae-b58a99bac70c None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:32.410083 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7311b2b-c10a-4406-b8ae-b58a99bac70c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:32.410391 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7311b2b-c10a-4406-b8ae-b58a99bac70c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:32.411809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7311b2b-c10a-4406-b8ae-b58a99bac70c None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:32.419933 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 228/448] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:09:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:32.425525 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7311b2b-c10a-4406-b8ae-b58a99bac70c None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:32.459200 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad2ae8b-5010-4b2e-8574-41daccf6e7fa None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:32.490641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:32.495157 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02306478-018a-4b09-8f20-448d83ffd682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:32.496030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02306478-018a-4b09-8f20-448d83ffd682 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:32.496312 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02306478-018a-4b09-8f20-448d83ffd682 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:32.496584 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02306478-018a-4b09-8f20-448d83ffd682 None admin] PATH_INFO: `/v3/projects/5e327e3411214877a9f427330ef40dc5` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:32.497817 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02306478-018a-4b09-8f20-448d83ffd682 None admin] RBAC: Authorizing `identity:get_project(project_id=5e327e3411214877a9f427330ef40dc5)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:32.499270 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02306478-018a-4b09-8f20-448d83ffd682 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:32.501006 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 221/449] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:09:32 2022] GET /identity/v3/projects/5e327e3411214877a9f427330ef40dc5 => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:32.506900 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02306478-018a-4b09-8f20-448d83ffd682 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:32.528717 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7311b2b-c10a-4406-b8ae-b58a99bac70c None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:32.564209 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df22c443-86e2-4e2b-801a-7e565ccc8378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:32.564857 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df22c443-86e2-4e2b-801a-7e565ccc8378 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:32.565163 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df22c443-86e2-4e2b-801a-7e565ccc8378 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:32.565406 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df22c443-86e2-4e2b-801a-7e565ccc8378 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:32.608736 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02306478-018a-4b09-8f20-448d83ffd682 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:32.622039 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:32.634073 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df22c443-86e2-4e2b-801a-7e565ccc8378 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:32.635553 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df22c443-86e2-4e2b-801a-7e565ccc8378 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:32.643785 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9407bbc4-c981-4e78-a03c-5296808899f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:32.644651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407bbc4-c981-4e78-a03c-5296808899f5 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:32.644934 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407bbc4-c981-4e78-a03c-5296808899f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:32.645237 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9407bbc4-c981-4e78-a03c-5296808899f5 None admin] PATH_INFO: `/v3/groups/fa3d562d2df749e18edd0819748e3e90` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:32.646547 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9407bbc4-c981-4e78-a03c-5296808899f5 None admin] RBAC: Authorizing `identity:get_group(group_id=fa3d562d2df749e18edd0819748e3e90)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:32.648334 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9407bbc4-c981-4e78-a03c-5296808899f5 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:32.650008 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 222/450] 10.222.0.50 () {56 vars in 1180 bytes} [Fri May 27 03:09:32 2022] GET /identity/v3/groups/fa3d562d2df749e18edd0819748e3e90 => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:32.655939 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9407bbc4-c981-4e78-a03c-5296808899f5 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:32.704970 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:32.746800 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 229/451] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:09:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 03:09:32.753366 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df22c443-86e2-4e2b-801a-7e565ccc8378 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:32.757323 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9407bbc4-c981-4e78-a03c-5296808899f5 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:32.792294 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3526ea-075a-4a82-aad1-6b856a9b1dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:32.792940 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3526ea-075a-4a82-aad1-6b856a9b1dab None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:32.793266 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3526ea-075a-4a82-aad1-6b856a9b1dab None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:32.793536 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3526ea-075a-4a82-aad1-6b856a9b1dab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:32.794366 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3526ea-075a-4a82-aad1-6b856a9b1dab None admin] RBAC: Adding query filter params (group.id=fa3d562d2df749e18edd0819748e3e90, role.id=9e63b73b3ce24bb3ad214ecdb6132696, scope.project.id=5e327e3411214877a9f427330ef40dc5) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:32.794674 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3526ea-075a-4a82-aad1-6b856a9b1dab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:32.796114 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3526ea-075a-4a82-aad1-6b856a9b1dab None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:32.802912 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 223/452] 10.222.0.50 () {56 vars in 1376 bytes} [Fri May 27 03:09:32 2022] GET /identity/v3/role_assignments?group.id=fa3d562d2df749e18edd0819748e3e90&scope.project.id=5e327e3411214877a9f427330ef40dc5&role.id=9e63b73b3ce24bb3ad214ecdb6132696 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:32.856882 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df22c443-86e2-4e2b-801a-7e565ccc8378 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:32.894603 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aff7df-a789-477b-a223-05e2c69a3712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:32.895480 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aff7df-a789-477b-a223-05e2c69a3712 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:32.895774 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aff7df-a789-477b-a223-05e2c69a3712 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:32.896060 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aff7df-a789-477b-a223-05e2c69a3712 None admin] PATH_INFO: `/v3/projects/73644351894d46fdb8889fca96b4c614` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:32.897355 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aff7df-a789-477b-a223-05e2c69a3712 None admin] RBAC: Authorizing `identity:get_project(project_id=73644351894d46fdb8889fca96b4c614)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:32.898845 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aff7df-a789-477b-a223-05e2c69a3712 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:32.900580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 230/453] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:09:32 2022] GET /identity/v3/projects/73644351894d46fdb8889fca96b4c614 => generated 344 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:32.906403 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3526ea-075a-4a82-aad1-6b856a9b1dab None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:32.956928 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:33.014822 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3526ea-075a-4a82-aad1-6b856a9b1dab None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:33.050070 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1607d7d1-9ee1-48af-9b44-2c6301c324c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:33.050938 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607d7d1-9ee1-48af-9b44-2c6301c324c5 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:33.051218 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607d7d1-9ee1-48af-9b44-2c6301c324c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:33.051517 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1607d7d1-9ee1-48af-9b44-2c6301c324c5 None admin] PATH_INFO: `/v3/groups/e78521d2f9ed4d93ad55d88822834e22` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:33.052932 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1607d7d1-9ee1-48af-9b44-2c6301c324c5 None admin] RBAC: Authorizing `identity:get_group(group_id=e78521d2f9ed4d93ad55d88822834e22)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:33.054421 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1607d7d1-9ee1-48af-9b44-2c6301c324c5 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:33.056064 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 224/454] 10.222.0.50 () {56 vars in 1180 bytes} [Fri May 27 03:09:32 2022] GET /identity/v3/groups/e78521d2f9ed4d93ad55d88822834e22 => generated 229 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:33.062136 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aff7df-a789-477b-a223-05e2c69a3712 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:33.097720 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:33.170008 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aff7df-a789-477b-a223-05e2c69a3712 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:33.206531 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0aec95-bb9b-4e43-8e19-fdf472fc7348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:33.207191 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0aec95-bb9b-4e43-8e19-fdf472fc7348 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:33.207469 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0aec95-bb9b-4e43-8e19-fdf472fc7348 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:33.207740 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0aec95-bb9b-4e43-8e19-fdf472fc7348 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:33.208940 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0aec95-bb9b-4e43-8e19-fdf472fc7348 None admin] RBAC: Adding query filter params (group.id=e78521d2f9ed4d93ad55d88822834e22, role.id=b4c524dd7bb44e94bd3b42d86e1cdf42, scope.project.id=73644351894d46fdb8889fca96b4c614) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:33.209331 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0aec95-bb9b-4e43-8e19-fdf472fc7348 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:33.210788 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0aec95-bb9b-4e43-8e19-fdf472fc7348 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:33.216610 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 231/455] 10.222.0.50 () {56 vars in 1376 bytes} [Fri May 27 03:09:33 2022] GET /identity/v3/role_assignments?group.id=e78521d2f9ed4d93ad55d88822834e22&scope.project.id=73644351894d46fdb8889fca96b4c614&role.id=b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 611 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:35.913276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4319e-d991-4e43-a749-b5a4b88e8780 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:35.914355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4319e-d991-4e43-a749-b5a4b88e8780 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:35.914837 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4319e-d991-4e43-a749-b5a4b88e8780 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:35.916743 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 225/456] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:35.921999 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ca85e3-0356-47f1-a627-a3e7bcaacd12 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:35.922818 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ca85e3-0356-47f1-a627-a3e7bcaacd12 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:35.923286 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ca85e3-0356-47f1-a627-a3e7bcaacd12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:35.947246 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-e9ca85e3-0356-47f1-a627-a3e7bcaacd12 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:35.948769 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-e9ca85e3-0356-47f1-a627-a3e7bcaacd12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:36.044940 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 232/457] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:36.080280 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a4319e-d991-4e43-a749-b5a4b88e8780 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:36.226167 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a4319e-d991-4e43-a749-b5a4b88e8780 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:36.242458 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:36.248287 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93893fc-3eeb-460a-b2a2-61feccd9d00b None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:36.248745 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93893fc-3eeb-460a-b2a2-61feccd9d00b None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:36.249055 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93893fc-3eeb-460a-b2a2-61feccd9d00b None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:36.252017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 233/458] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:36.257433 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676830a9-bb8b-4437-b5e9-2409970c7aa8 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:36.257748 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676830a9-bb8b-4437-b5e9-2409970c7aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:36.258115 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676830a9-bb8b-4437-b5e9-2409970c7aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:36.277155 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b9c8f-9a7c-4eb7-a227-ec73b52b2c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:36.277925 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b9c8f-9a7c-4eb7-a227-ec73b52b2c70 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:36.278203 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b9c8f-9a7c-4eb7-a227-ec73b52b2c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:36.278479 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b9c8f-9a7c-4eb7-a227-ec73b52b2c70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:36.285696 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-676830a9-bb8b-4437-b5e9-2409970c7aa8 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:36.287558 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-676830a9-bb8b-4437-b5e9-2409970c7aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:36.295843 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b9c8f-9a7c-4eb7-a227-ec73b52b2c70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:36.297565 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b9c8f-9a7c-4eb7-a227-ec73b52b2c70 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:36.315413 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-f67b9c8f-9a7c-4eb7-a227-ec73b52b2c70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:36.315413 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:36.315413 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:36.315413 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:36.315413 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:36.315413 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:36.315413 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:36.315413 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:36.315413 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:36.315413 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:36.315413 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:36.315413 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:36.315413 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:09:36.315413 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:09:36.315413 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:09:36.316496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:09:36.316496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:36.316496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:36.316496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:36.316496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:36.316496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:36.316496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:36.316496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:36.316496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:36.316496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:36.316496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:36.316496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:36.316496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:36.316496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:36.316496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:36.316496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:36.316496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:36.317753 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:09:36.317753 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:09:36.317753 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:36.317753 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:36.317753 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 226/459] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:36.324245 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60161a-04dd-48da-a743-39fef23cd197 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:36.324245 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60161a-04dd-48da-a743-39fef23cd197 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:36.324245 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60161a-04dd-48da-a743-39fef23cd197 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:36.324245 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 227/460] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:36.325704 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60161a-04dd-48da-a743-39fef23cd197 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:36.326199 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:36.382977 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 234/461] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:09:36.384730 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095201b-b92e-4e3b-a92c-e13fc5c8c50d None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:36.385226 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095201b-b92e-4e3b-a92c-e13fc5c8c50d None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:36.385506 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3095201b-b92e-4e3b-a92c-e13fc5c8c50d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:36.406311 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-3095201b-b92e-4e3b-a92c-e13fc5c8c50d None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:36.407635 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-3095201b-b92e-4e3b-a92c-e13fc5c8c50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:36.456335 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60161a-04dd-48da-a743-39fef23cd197 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:36.489173 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 235/462] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:36.490256 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3095201b-b92e-4e3b-a92c-e13fc5c8c50d None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:36.505205 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9748080-be4f-4fd4-b7a7-d3fa7d59d4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:36.506340 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9748080-be4f-4fd4-b7a7-d3fa7d59d4fe None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:36.506807 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9748080-be4f-4fd4-b7a7-d3fa7d59d4fe None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:36.507268 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9748080-be4f-4fd4-b7a7-d3fa7d59d4fe None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:36.509801 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9748080-be4f-4fd4-b7a7-d3fa7d59d4fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:36.510321 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9748080-be4f-4fd4-b7a7-d3fa7d59d4fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:36.511974 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9748080-be4f-4fd4-b7a7-d3fa7d59d4fe None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:36.530807 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 228/463] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:09:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:36.531617 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9748080-be4f-4fd4-b7a7-d3fa7d59d4fe None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:36.559157 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:36.594609 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:36.608502 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3095201b-b92e-4e3b-a92c-e13fc5c8c50d None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:36.643717 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9748080-be4f-4fd4-b7a7-d3fa7d59d4fe None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:36.644941 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81427aa-8682-4634-bb03-dcf577d1c41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:36.646243 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81427aa-8682-4634-bb03-dcf577d1c41e None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:36.646243 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81427aa-8682-4634-bb03-dcf577d1c41e None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:36.647259 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81427aa-8682-4634-bb03-dcf577d1c41e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:36.657455 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81427aa-8682-4634-bb03-dcf577d1c41e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:36.659079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81427aa-8682-4634-bb03-dcf577d1c41e None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:36.678104 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-a81427aa-8682-4634-bb03-dcf577d1c41e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:36.678104 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:36.678104 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:36.678104 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:36.678104 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:36.678104 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:36.678104 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:36.678104 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:36.678104 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:36.678104 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:36.678104 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:36.678104 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:36.678104 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:09:36.678104 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:09:36.678104 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:09:36.679696 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:09:36.679696 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:36.679696 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:36.679696 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:36.679696 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:36.679696 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:36.679696 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:36.679696 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:36.679696 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:36.679696 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:36.679696 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:36.679696 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:36.679696 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:36.679696 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:36.679696 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:36.679696 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:36.679696 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:36.680558 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:09:36.680558 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:09:36.680558 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:36.680558 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:36.680558 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 236/464] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:36.680558 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81427aa-8682-4634-bb03-dcf577d1c41e None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:36.685968 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f90a02-ecf4-4c28-8add-872b1dddb460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:36.686784 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f90a02-ecf4-4c28-8add-872b1dddb460 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:36.687066 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f90a02-ecf4-4c28-8add-872b1dddb460 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:36.687342 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f90a02-ecf4-4c28-8add-872b1dddb460 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:36.699094 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f90a02-ecf4-4c28-8add-872b1dddb460 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:36.700534 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f90a02-ecf4-4c28-8add-872b1dddb460 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:36.718673 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-04f90a02-ecf4-4c28-8add-872b1dddb460 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:36.718673 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:36.718673 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:36.718673 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:36.718673 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:36.718673 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:36.718673 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:36.718673 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:36.718673 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:36.718673 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:36.718673 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:36.718673 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:36.718673 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:09:36.718673 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:09:36.718673 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:09:36.719535 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:09:36.719535 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:36.719535 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:36.719535 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:36.719535 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:36.719535 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:36.719535 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:36.719535 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:36.719535 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:36.719535 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:36.719535 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:36.719535 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:36.719535 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:36.719535 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:36.719535 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:36.719535 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:36.719535 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:36.720346 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:09:36.720346 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:09:36.720346 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:36.720346 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:36.720346 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 229/465] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:36.721308 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d44b1e-6bcc-47b2-a6bb-d4c2b11c4a66 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:36.721592 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d44b1e-6bcc-47b2-a6bb-d4c2b11c4a66 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:36.721868 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d44b1e-6bcc-47b2-a6bb-d4c2b11c4a66 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:36.725000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 230/466] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:36.725000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d44b1e-6bcc-47b2-a6bb-d4c2b11c4a66 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:36.789746 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81427aa-8682-4634-bb03-dcf577d1c41e None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:36.804679 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:36.827115 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178cd93f-e027-43e1-a7c0-1131773d15f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:36.827792 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178cd93f-e027-43e1-a7c0-1131773d15f5 None admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:36.828065 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178cd93f-e027-43e1-a7c0-1131773d15f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:36.828334 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178cd93f-e027-43e1-a7c0-1131773d15f5 None admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:36.829068 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178cd93f-e027-43e1-a7c0-1131773d15f5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:36.830514 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178cd93f-e027-43e1-a7c0-1131773d15f5 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:36.831407 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d44b1e-6bcc-47b2-a6bb-d4c2b11c4a66 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:36.840303 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:36.855665 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 237/467] 10.222.0.50 () {60 vars in 1132 bytes} [Fri May 27 03:09:36 2022] POST /identity/v3/users => generated 246 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:09:36.856813 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178cd93f-e027-43e1-a7c0-1131773d15f5 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:36.870958 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e22c8a1-67c7-404a-b25c-570690bc63fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:36.871938 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22c8a1-67c7-404a-b25c-570690bc63fb None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:36.872501 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22c8a1-67c7-404a-b25c-570690bc63fb None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:36.873044 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e22c8a1-67c7-404a-b25c-570690bc63fb None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:36.873790 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e22c8a1-67c7-404a-b25c-570690bc63fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:36.874103 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e22c8a1-67c7-404a-b25c-570690bc63fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:36.875564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e22c8a1-67c7-404a-b25c-570690bc63fb None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:36.890175 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 231/468] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:09:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:36.896629 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841ab1f-c38f-45dd-be12-42edcf9c0d1d None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:36.896945 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841ab1f-c38f-45dd-be12-42edcf9c0d1d None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:36.897213 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841ab1f-c38f-45dd-be12-42edcf9c0d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:36.920190 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-e841ab1f-c38f-45dd-be12-42edcf9c0d1d None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:36.921517 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-e841ab1f-c38f-45dd-be12-42edcf9c0d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:37.008977 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 232/469] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:37.008977 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e841ab1f-c38f-45dd-be12-42edcf9c0d1d None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:37.012502 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178cd93f-e027-43e1-a7c0-1131773d15f5 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:37.060074 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f6dd04-dd5a-496a-bfea-5cef45effcd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:37.060660 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6dd04-dd5a-496a-bfea-5cef45effcd0 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:37.060964 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6dd04-dd5a-496a-bfea-5cef45effcd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:37.061243 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6dd04-dd5a-496a-bfea-5cef45effcd0 None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:37.061985 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f6dd04-dd5a-496a-bfea-5cef45effcd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:37.062290 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f6dd04-dd5a-496a-bfea-5cef45effcd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:37.063683 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f6dd04-dd5a-496a-bfea-5cef45effcd0 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:37.083241 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 238/470] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:09:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:37.084224 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f6dd04-dd5a-496a-bfea-5cef45effcd0 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:37.095857 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:37.134725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e841ab1f-c38f-45dd-be12-42edcf9c0d1d None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:37.145979 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:37.185325 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea859f-a99a-4aa1-bfba-e9be9c65b3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:37.186026 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea859f-a99a-4aa1-bfba-e9be9c65b3cf None admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:37.186302 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea859f-a99a-4aa1-bfba-e9be9c65b3cf None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:37.186572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea859f-a99a-4aa1-bfba-e9be9c65b3cf None admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:37.187279 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea859f-a99a-4aa1-bfba-e9be9c65b3cf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:37.188724 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea859f-a99a-4aa1-bfba-e9be9c65b3cf None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:37.202277 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f6dd04-dd5a-496a-bfea-5cef45effcd0 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:37.215558 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 233/471] 10.222.0.50 () {60 vars in 1132 bytes} [Fri May 27 03:09:37 2022] POST /identity/v3/users => generated 248 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:09:37.217158 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea859f-a99a-4aa1-bfba-e9be9c65b3cf None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:37.251651 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd1d1e-1470-4599-81e6-c6f2e8b78c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:37.252855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd1d1e-1470-4599-81e6-c6f2e8b78c9a None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:37.253402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd1d1e-1470-4599-81e6-c6f2e8b78c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:37.253844 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd1d1e-1470-4599-81e6-c6f2e8b78c9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:37.266086 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acd1d1e-1470-4599-81e6-c6f2e8b78c9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:37.268015 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acd1d1e-1470-4599-81e6-c6f2e8b78c9a None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:37.298085 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-7acd1d1e-1470-4599-81e6-c6f2e8b78c9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:37.298085 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:37.298085 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:37.298085 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:37.298085 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:37.298085 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:37.298085 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:37.298085 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:37.298085 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:37.298085 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:37.298085 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:37.298085 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:37.298085 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:09:37.298085 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:09:37.298085 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:09:37.298892 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:09:37.298892 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:37.298892 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:37.298892 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:37.298892 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:37.298892 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:37.298892 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:37.298892 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:37.298892 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:37.298892 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:37.298892 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:37.298892 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:37.298892 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:37.298892 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:37.298892 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:37.298892 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:37.298892 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:37.299716 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:09:37.299716 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:09:37.299716 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:37.299716 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:37.301910 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 239/472] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:37.314801 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd1d1e-1470-4599-81e6-c6f2e8b78c9a None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:37.344004 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea859f-a99a-4aa1-bfba-e9be9c65b3cf None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:37.371536 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:37.385950 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7637b05-f301-42f2-b146-71f7cd8fcfe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:37.386370 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7637b05-f301-42f2-b146-71f7cd8fcfe3 None admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:37.386889 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7637b05-f301-42f2-b146-71f7cd8fcfe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:37.388004 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7637b05-f301-42f2-b146-71f7cd8fcfe3 None admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:37.389083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7637b05-f301-42f2-b146-71f7cd8fcfe3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:37.390683 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7637b05-f301-42f2-b146-71f7cd8fcfe3 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:37.419025 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 234/473] 10.222.0.50 () {60 vars in 1132 bytes} [Fri May 27 03:09:37 2022] POST /identity/v3/users => generated 248 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:09:37.488390 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd1d1e-1470-4599-81e6-c6f2e8b78c9a None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:37.500534 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:37.531127 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b73bdb9-904c-4eaf-ad4d-752d6f474cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:37.531820 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73bdb9-904c-4eaf-ad4d-752d6f474cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:37.532095 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73bdb9-904c-4eaf-ad4d-752d6f474cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:37.532367 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73bdb9-904c-4eaf-ad4d-752d6f474cd8 None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:37.533138 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b73bdb9-904c-4eaf-ad4d-752d6f474cd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:37.533442 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b73bdb9-904c-4eaf-ad4d-752d6f474cd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:37.535310 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b73bdb9-904c-4eaf-ad4d-752d6f474cd8 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:37.553414 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 240/474] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:09:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:37.556580 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7637b05-f301-42f2-b146-71f7cd8fcfe3 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:37.652799 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3cda3-07a3-4c0f-9eac-a0f2df31709b None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:37.653002 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3cda3-07a3-4c0f-9eac-a0f2df31709b None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:37.653269 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee3cda3-07a3-4c0f-9eac-a0f2df31709b None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:37.654773 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 241/475] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:09:37.662820 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e28e73b-18de-4fc2-a7e8-d279e6525666 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:37.663385 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e28e73b-18de-4fc2-a7e8-d279e6525666 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:37.663837 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e28e73b-18de-4fc2-a7e8-d279e6525666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:37.683235 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7637b05-f301-42f2-b146-71f7cd8fcfe3 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:37.690433 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-7e28e73b-18de-4fc2-a7e8-d279e6525666 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:37.691641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-7e28e73b-18de-4fc2-a7e8-d279e6525666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:37.735505 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e605b0-7af6-432c-99ab-72c6333f0f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:37.736555 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e605b0-7af6-432c-99ab-72c6333f0f4d None admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:37.737037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e605b0-7af6-432c-99ab-72c6333f0f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:37.737494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e605b0-7af6-432c-99ab-72c6333f0f4d None admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:37.738220 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e605b0-7af6-432c-99ab-72c6333f0f4d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:37.739741 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e605b0-7af6-432c-99ab-72c6333f0f4d None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:37.767915 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 235/476] 10.222.0.50 () {60 vars in 1132 bytes} [Fri May 27 03:09:37 2022] POST /identity/v3/users => generated 249 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:09:37.790198 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 242/477] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:37.813868 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e605b0-7af6-432c-99ab-72c6333f0f4d None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:37.855176 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:37.934737 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e605b0-7af6-432c-99ab-72c6333f0f4d None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:37.972550 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda4710-34d3-49a1-9763-9f1a9caeddde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:37.973261 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda4710-34d3-49a1-9763-9f1a9caeddde None admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:37.973623 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda4710-34d3-49a1-9763-9f1a9caeddde None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:37.974107 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda4710-34d3-49a1-9763-9f1a9caeddde None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:37.975350 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda4710-34d3-49a1-9763-9f1a9caeddde None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:37.976798 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda4710-34d3-49a1-9763-9f1a9caeddde None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:37.984868 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.sql.core [None req-5dda4710-34d3-49a1-9763-9f1a9caeddde None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 27 03:09:37.984868 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 27 03:09:37.984868 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [parameters: {'id': '0b0f292629ed4b2f92d22389b88ff750', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 27 03:09:37.984868 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80410) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 27 03:09:38.003000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-5dda4710-34d3-49a1-9763-9f1a9caeddde None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 27 03:09:38.003000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:38.003000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 03:09:38.003000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 03:09:38.003000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 03:09:38.003000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 03:09:38.003000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 03:09:38.003000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application result = self._query(query) May 27 03:09:38.003000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 03:09:38.003000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application conn.query(q) May 27 03:09:38.003000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 03:09:38.003000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 03:09:38.003000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 03:09:38.003000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application result.read() May 27 03:09:38.003000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 03:09:38.003996 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 03:09:38.003996 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 03:09:38.003996 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 03:09:38.003996 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 03:09:38.003996 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 03:09:38.003996 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 03:09:38.003996 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 03:09:38.003996 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 27 03:09:38.003996 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:38.003996 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 27 03:09:38.003996 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:38.003996 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:38.003996 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 27 03:09:38.003996 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 27 03:09:38.003996 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 27 03:09:38.003996 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return ref.to_dict() May 27 03:09:38.003996 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 03:09:38.003996 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application next(self.gen) May 27 03:09:38.005452 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 27 03:09:38.005452 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application yield resource May 27 03:09:38.005452 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 03:09:38.005452 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application next(self.gen) May 27 03:09:38.005452 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 27 03:09:38.005452 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self.session.rollback() May 27 03:09:38.005452 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 27 03:09:38.005452 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self.force_reraise() May 27 03:09:38.005452 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 27 03:09:38.005452 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise self.value May 27 03:09:38.005452 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 27 03:09:38.005452 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 27 03:09:38.005452 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 27 03:09:38.005452 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application session.commit() May 27 03:09:38.005452 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 27 03:09:38.005452 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 27 03:09:38.005452 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 27 03:09:38.006367 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self._prepare_impl() May 27 03:09:38.006367 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 27 03:09:38.006367 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self.session.flush() May 27 03:09:38.006367 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 27 03:09:38.006367 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self._flush(objects) May 27 03:09:38.006367 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 27 03:09:38.006367 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 27 03:09:38.006367 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 27 03:09:38.006367 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application compat.raise_( May 27 03:09:38.006367 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 03:09:38.006367 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception May 27 03:09:38.006367 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 27 03:09:38.006367 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application flush_context.execute() May 27 03:09:38.006367 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 27 03:09:38.006367 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rec.execute(self) May 27 03:09:38.006367 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 27 03:09:38.006367 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 27 03:09:38.007215 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 27 03:09:38.007215 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application _emit_insert_statements( May 27 03:09:38.007215 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 27 03:09:38.007215 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application c = connection._execute_20( May 27 03:09:38.007215 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 27 03:09:38.007215 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 27 03:09:38.007215 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 27 03:09:38.007215 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 27 03:09:38.007215 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 27 03:09:38.007215 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ret = self._execute_context( May 27 03:09:38.007215 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 27 03:09:38.007215 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 27 03:09:38.007215 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 27 03:09:38.007215 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 27 03:09:38.007215 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 03:09:38.007215 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception May 27 03:09:38.008020 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 03:09:38.008020 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 03:09:38.008020 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 03:09:38.008020 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 03:09:38.008020 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 03:09:38.008020 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application result = self._query(query) May 27 03:09:38.008020 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 03:09:38.008020 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application conn.query(q) May 27 03:09:38.008020 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 03:09:38.008020 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 03:09:38.008020 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 03:09:38.008020 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application result.read() May 27 03:09:38.008020 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 03:09:38.008020 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 03:09:38.008020 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 03:09:38.008020 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 03:09:38.008779 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 03:09:38.008779 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 03:09:38.008779 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 03:09:38.008779 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 03:09:38.008779 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 27 03:09:38.008779 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 27 03:09:38.008779 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [parameters: {'id': '0b0f292629ed4b2f92d22389b88ff750', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 27 03:09:38.008779 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 27 03:09:38.008779 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:38.008779 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:09:38.008779 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:38.008779 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:38.008779 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:38.008779 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:38.008779 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:38.008779 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:38.011761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:38.011761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:38.011761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:38.011761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:38.011761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:38.011761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:38.011761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 27 03:09:38.011761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 27 03:09:38.011761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:38.011761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:38.011761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 27 03:09:38.011761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 27 03:09:38.011761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 27 03:09:38.011761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 27 03:09:38.011761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 27 03:09:38.011761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:38.013735 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 236/478] 10.222.0.50 () {60 vars in 1131 bytes} [Fri May 27 03:09:37 2022] POST /identity/v3/roles => generated 144 bytes in 191 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:38.014242 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e28e73b-18de-4fc2-a7e8-d279e6525666 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:38.134772 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:38.139388 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e28e73b-18de-4fc2-a7e8-d279e6525666 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:38.179258 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232c6910-a2ae-4c23-87b0-3f20564804a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:38.180080 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c6910-a2ae-4c23-87b0-3f20564804a7 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:38.180354 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c6910-a2ae-4c23-87b0-3f20564804a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:38.180672 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c6910-a2ae-4c23-87b0-3f20564804a7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:38.193264 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232c6910-a2ae-4c23-87b0-3f20564804a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:38.201746 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-232c6910-a2ae-4c23-87b0-3f20564804a7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:09:38.201746 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:38.201746 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:38.201746 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:38.201746 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:38.201746 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:38.201746 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:38.201746 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:38.201746 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:38.201746 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:38.201746 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:38.201746 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:38.201746 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:38.201746 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:38.201746 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:38.201746 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:38.202686 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:38.202686 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:38.202686 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:38.202686 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:38.202686 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:38.202686 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:38.202686 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:38.202686 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:38.202686 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:38.202686 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:38.202686 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:38.202686 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:38.202686 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:38.202686 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:38.202686 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:38.202686 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:38.203462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:38.203462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:38.203462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:38.203462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:38.203462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:38.203462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:38.203462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:38.203462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:38.203462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:38.203462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:38.203462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 03:09:38.203462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:38.203462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 243/479] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 03:09:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:38.209276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda4710-34d3-49a1-9763-9f1a9caeddde None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:38.335810 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:38.339725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda4710-34d3-49a1-9763-9f1a9caeddde None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:38.376913 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2be8316-e167-4e83-bd2a-19df61f3eb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:38.377616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be8316-e167-4e83-bd2a-19df61f3eb8c None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:38.377904 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be8316-e167-4e83-bd2a-19df61f3eb8c None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:38.378172 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2be8316-e167-4e83-bd2a-19df61f3eb8c None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:38.378998 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2be8316-e167-4e83-bd2a-19df61f3eb8c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:38.379307 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2be8316-e167-4e83-bd2a-19df61f3eb8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:38.381217 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2be8316-e167-4e83-bd2a-19df61f3eb8c None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:38.390072 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 237/480] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:09:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:39.465692 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062f1e8-5979-4be6-9609-57588bf4b502 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:39.466424 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062f1e8-5979-4be6-9609-57588bf4b502 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:39.466424 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062f1e8-5979-4be6-9609-57588bf4b502 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:39.467670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 244/481] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:39.473270 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597096d2-93ae-42cf-96f6-1ad510fca4b5 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:39.473839 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597096d2-93ae-42cf-96f6-1ad510fca4b5 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:39.474095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597096d2-93ae-42cf-96f6-1ad510fca4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:39.502340 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-597096d2-93ae-42cf-96f6-1ad510fca4b5 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:39.503535 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-597096d2-93ae-42cf-96f6-1ad510fca4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:39.603295 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 238/482] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:09:39.632992 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062f1e8-5979-4be6-9609-57588bf4b502 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:39.766001 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062f1e8-5979-4be6-9609-57588bf4b502 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:39.807698 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74754b-c567-441c-a54e-f0220837ef13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:39.808901 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74754b-c567-441c-a54e-f0220837ef13 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:39.809396 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74754b-c567-441c-a54e-f0220837ef13 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:39.809871 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74754b-c567-441c-a54e-f0220837ef13 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:39.824874 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74754b-c567-441c-a54e-f0220837ef13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:39.832678 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-cd74754b-c567-441c-a54e-f0220837ef13 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:09:39.832678 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:39.832678 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:39.832678 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:39.832678 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:39.832678 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:39.832678 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:39.832678 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:39.832678 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:39.832678 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:39.832678 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:39.832678 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:39.832678 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:39.832678 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:39.832678 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:39.832678 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:39.833877 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:39.833877 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:39.833877 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:39.833877 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:39.833877 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:39.833877 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:39.833877 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:39.833877 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:39.833877 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:39.833877 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:39.833877 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:39.833877 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:39.833877 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:39.833877 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:39.833877 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:39.833877 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:39.834860 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:39.834860 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:39.834860 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:39.834860 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:39.834860 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:39.834860 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:39.834860 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:39.834860 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:39.834860 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:39.834860 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:39.834860 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:09:39.834860 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:39.841232 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 245/483] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 03:09:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:39.843224 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597096d2-93ae-42cf-96f6-1ad510fca4b5 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:39.968032 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:39.979572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597096d2-93ae-42cf-96f6-1ad510fca4b5 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:40.027289 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dceb46-0aff-4f36-a997-95a1a602620d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:40.027994 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dceb46-0aff-4f36-a997-95a1a602620d None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:40.028317 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dceb46-0aff-4f36-a997-95a1a602620d None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:40.028595 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dceb46-0aff-4f36-a997-95a1a602620d None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:40.029401 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dceb46-0aff-4f36-a997-95a1a602620d None admin] RBAC: Adding query filter params (name=service) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:40.029707 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dceb46-0aff-4f36-a997-95a1a602620d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:40.031215 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dceb46-0aff-4f36-a997-95a1a602620d None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:40.045881 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 239/484] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 03:09:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:40.051565 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74754b-c567-441c-a54e-f0220837ef13 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:40.087678 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:40.101888 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f956c-3730-4106-a44b-6d228052f9df None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:40.102196 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f956c-3730-4106-a44b-6d228052f9df None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:40.102454 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f956c-3730-4106-a44b-6d228052f9df None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:40.103961 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 240/485] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:40.110641 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9882db0-329e-424d-97dd-62b4255f8ed4 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:40.111135 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9882db0-329e-424d-97dd-62b4255f8ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:40.111600 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9882db0-329e-424d-97dd-62b4255f8ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:40.130919 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-c9882db0-329e-424d-97dd-62b4255f8ed4 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:40.132451 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-c9882db0-329e-424d-97dd-62b4255f8ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:40.210703 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 241/486] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:09:40.236781 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9882db0-329e-424d-97dd-62b4255f8ed4 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:40.257413 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74754b-c567-441c-a54e-f0220837ef13 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:40.320837 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e8104-59f1-476b-a41e-567453827b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:40.321714 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e8104-59f1-476b-a41e-567453827b58 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:40.321991 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e8104-59f1-476b-a41e-567453827b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:40.322286 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e8104-59f1-476b-a41e-567453827b58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:40.337092 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e8104-59f1-476b-a41e-567453827b58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:40.338585 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e8104-59f1-476b-a41e-567453827b58 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:40.356248 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-021e8104-59f1-476b-a41e-567453827b58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:40.356248 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:40.356248 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:40.356248 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:40.356248 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:40.356248 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:40.356248 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:40.356248 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:40.356248 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:40.356248 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:40.356248 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:40.356248 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:40.356248 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:09:40.356248 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:09:40.356248 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:09:40.357184 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:09:40.357184 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:40.357184 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:40.357184 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:40.357184 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:40.357184 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:40.357184 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:40.357184 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:40.357184 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:40.357184 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:40.357184 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:40.357184 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:40.357184 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:40.357184 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:40.357184 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:40.357184 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:40.357184 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:40.357989 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:09:40.357989 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:09:40.357989 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:40.357989 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:40.357989 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 246/487] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:40.360987 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79738be0-015d-4237-813b-38dcf3541430 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:40.360987 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79738be0-015d-4237-813b-38dcf3541430 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:40.360987 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79738be0-015d-4237-813b-38dcf3541430 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:40.360987 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 247/488] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:40.367121 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79738be0-015d-4237-813b-38dcf3541430 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:40.376201 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:40.400144 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9882db0-329e-424d-97dd-62b4255f8ed4 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:40.436615 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c795a-0d4e-48f7-88bc-7f85a5f31886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:40.437457 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c795a-0d4e-48f7-88bc-7f85a5f31886 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:40.437791 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c795a-0d4e-48f7-88bc-7f85a5f31886 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:40.438064 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c795a-0d4e-48f7-88bc-7f85a5f31886 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:40.457095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c795a-0d4e-48f7-88bc-7f85a5f31886 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:40.465824 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-5b2c795a-0d4e-48f7-88bc-7f85a5f31886 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:09:40.465824 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:40.465824 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:40.465824 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:40.465824 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:40.465824 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:40.465824 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:40.465824 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:40.465824 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:40.465824 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:40.465824 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:40.465824 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:40.465824 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:40.465824 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:40.465824 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:40.465824 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:40.466725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:40.466725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:40.466725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:40.466725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:40.466725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:40.466725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:40.466725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:40.466725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:40.466725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:40.466725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:40.466725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:40.466725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:40.466725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:40.466725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:40.466725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:40.466725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:40.467497 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:40.467497 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:40.467497 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:40.467497 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:40.467497 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:40.467497 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:40.467497 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:40.467497 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:40.467497 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:40.467497 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:40.467497 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:09:40.467497 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:40.467497 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 242/489] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 03:09:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:09:40.468711 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f82cc-ed04-483a-98e4-84d3afe22efe None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:40.469026 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f82cc-ed04-483a-98e4-84d3afe22efe None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:40.469320 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f82cc-ed04-483a-98e4-84d3afe22efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:40.481571 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:40.488427 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-e69f82cc-ed04-483a-98e4-84d3afe22efe None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:40.490504 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-e69f82cc-ed04-483a-98e4-84d3afe22efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:40.526264 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79738be0-015d-4237-813b-38dcf3541430 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:40.570244 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 243/490] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:40.570244 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f82cc-ed04-483a-98e4-84d3afe22efe None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:40.575756 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84794a-f87e-4720-9ab2-493d8defbfaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:40.577235 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84794a-f87e-4720-9ab2-493d8defbfaf None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:40.577722 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84794a-f87e-4720-9ab2-493d8defbfaf None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:40.578198 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84794a-f87e-4720-9ab2-493d8defbfaf None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:40.584982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c84794a-f87e-4720-9ab2-493d8defbfaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:40.584982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c84794a-f87e-4720-9ab2-493d8defbfaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:40.584982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c84794a-f87e-4720-9ab2-493d8defbfaf None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:40.609726 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 248/491] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:09:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:40.611041 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84794a-f87e-4720-9ab2-493d8defbfaf None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:40.679466 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:40.721611 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f82cc-ed04-483a-98e4-84d3afe22efe None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:40.733005 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84794a-f87e-4720-9ab2-493d8defbfaf None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:40.760209 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ccd53-fd90-45fa-8470-8eb89f63fd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:40.760897 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ccd53-fd90-45fa-8470-8eb89f63fd48 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:40.761211 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ccd53-fd90-45fa-8470-8eb89f63fd48 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:40.761487 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ccd53-fd90-45fa-8470-8eb89f63fd48 None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:40.762374 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ccd53-fd90-45fa-8470-8eb89f63fd48 None admin] RBAC: Adding query filter params (name=service) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:40.762638 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ccd53-fd90-45fa-8470-8eb89f63fd48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:40.764056 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ccd53-fd90-45fa-8470-8eb89f63fd48 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:40.765149 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:40.771264 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867db5df-3fb8-4b18-9efd-f16fd3c9f56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:40.772403 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867db5df-3fb8-4b18-9efd-f16fd3c9f56b None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:40.772895 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867db5df-3fb8-4b18-9efd-f16fd3c9f56b None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:40.773444 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867db5df-3fb8-4b18-9efd-f16fd3c9f56b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:40.776302 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 244/492] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 03:09:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:40.777412 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ccd53-fd90-45fa-8470-8eb89f63fd48 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:40.785884 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867db5df-3fb8-4b18-9efd-f16fd3c9f56b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:40.794093 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-867db5df-3fb8-4b18-9efd-f16fd3c9f56b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:09:40.794093 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:40.794093 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:40.794093 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:40.794093 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:40.794093 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:40.794093 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:40.794093 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:40.794093 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:40.794093 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:40.794093 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:40.794093 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:40.794093 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:40.794093 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:40.794093 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:40.794093 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:40.794953 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:40.794953 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:40.794953 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:40.794953 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:40.794953 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:40.794953 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:40.794953 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:40.794953 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:40.794953 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:40.794953 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:40.794953 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:40.794953 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:40.794953 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:40.794953 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:40.794953 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:40.794953 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:40.795727 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:40.795727 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:40.795727 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:40.795727 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:40.795727 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:40.795727 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:40.795727 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:40.795727 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:40.795727 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:40.795727 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:40.795727 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:09:40.795727 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:40.798142 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 249/493] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 03:09:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:40.803709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867db5df-3fb8-4b18-9efd-f16fd3c9f56b None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:40.888153 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ccd53-fd90-45fa-8470-8eb89f63fd48 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:40.912270 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867db5df-3fb8-4b18-9efd-f16fd3c9f56b None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:40.924692 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41244e81-5010-461c-96a0-ac48200be322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:40.925623 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41244e81-5010-461c-96a0-ac48200be322 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:40.925901 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41244e81-5010-461c-96a0-ac48200be322 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:40.926275 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41244e81-5010-461c-96a0-ac48200be322 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:40.928859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:40.935213 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41244e81-5010-461c-96a0-ac48200be322 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:40.937175 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41244e81-5010-461c-96a0-ac48200be322 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:40.947143 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3521cf4-2ad0-4b94-81bb-5fc1ec063a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:40.947948 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3521cf4-2ad0-4b94-81bb-5fc1ec063a21 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:40.948223 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3521cf4-2ad0-4b94-81bb-5fc1ec063a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:40.948490 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3521cf4-2ad0-4b94-81bb-5fc1ec063a21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:40.954253 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-41244e81-5010-461c-96a0-ac48200be322 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 03:09:40.954253 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:40.954253 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:40.954253 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:40.954253 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:40.954253 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:40.954253 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:40.954253 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:40.954253 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:40.954253 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:40.954253 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:40.954253 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:40.954253 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:09:40.954253 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:09:40.954253 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:09:40.954253 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:09:40.955202 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:40.955202 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:40.955202 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:09:40.955202 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:09:40.955202 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:09:40.955202 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:09:40.955202 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:40.955202 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:40.955202 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:40.955202 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:40.955202 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:40.955202 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:40.955202 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:40.955202 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:40.955202 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:40.955202 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:40.955202 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:40.955202 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:40.956080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:40.956080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:40.956080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:09:40.956080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:09:40.956080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:09:40.956080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:09:40.956080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:09:40.956080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:09:40.956080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 03:09:40.956080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:40.956080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 245/494] 10.222.0.50 () {56 vars in 1093 bytes} [Fri May 27 03:09:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:40.957416 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41244e81-5010-461c-96a0-ac48200be322 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:40.964411 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3521cf4-2ad0-4b94-81bb-5fc1ec063a21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:40.965926 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3521cf4-2ad0-4b94-81bb-5fc1ec063a21 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:40.983613 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-b3521cf4-2ad0-4b94-81bb-5fc1ec063a21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:40.983613 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:40.983613 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:40.983613 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:40.983613 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:40.983613 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:40.983613 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:40.983613 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:40.983613 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:40.983613 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:40.983613 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:40.983613 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:40.983613 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:09:40.983613 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:09:40.983613 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:09:40.984484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:09:40.984484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:40.984484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:40.984484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:40.984484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:40.984484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:40.984484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:40.984484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:40.984484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:40.984484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:40.984484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:40.984484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:40.984484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:40.984484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:40.984484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:40.984484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:40.984484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:40.985355 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:09:40.985355 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:09:40.985355 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:40.985355 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:40.989538 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 250/495] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:40.990843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf116a-df99-48e0-b975-bd98de86985c None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:40.991125 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf116a-df99-48e0-b975-bd98de86985c None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:40.991397 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf116a-df99-48e0-b975-bd98de86985c None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:40.992690 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 251/496] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:40.996271 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf116a-df99-48e0-b975-bd98de86985c None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:41.048246 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:41.077347 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41244e81-5010-461c-96a0-ac48200be322 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:41.110185 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf116a-df99-48e0-b975-bd98de86985c None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:41.115422 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ecbd14-3c86-4814-b083-0632ac2becaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:41.116107 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ecbd14-3c86-4814-b083-0632ac2becaa None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:41.116385 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ecbd14-3c86-4814-b083-0632ac2becaa None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:41.116655 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ecbd14-3c86-4814-b083-0632ac2becaa None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:41.117574 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ecbd14-3c86-4814-b083-0632ac2becaa None admin] RBAC: Adding query filter params (name=service) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:41.117881 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ecbd14-3c86-4814-b083-0632ac2becaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:41.119290 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ecbd14-3c86-4814-b083-0632ac2becaa None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:41.133039 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 246/497] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 03:09:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:41.133039 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ecbd14-3c86-4814-b083-0632ac2becaa None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:41.147889 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f563e66c-573d-4f9c-b4ab-37a39c036bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:41.148568 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563e66c-573d-4f9c-b4ab-37a39c036bdb None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:41.148856 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563e66c-573d-4f9c-b4ab-37a39c036bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:41.149170 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563e66c-573d-4f9c-b4ab-37a39c036bdb None admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:41.150005 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f563e66c-573d-4f9c-b4ab-37a39c036bdb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:41.150313 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f563e66c-573d-4f9c-b4ab-37a39c036bdb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:41.151769 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f563e66c-573d-4f9c-b4ab-37a39c036bdb None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:41.188693 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 252/498] 10.222.0.50 () {56 vars in 1133 bytes} [Fri May 27 03:09:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:41.190089 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ded3a-b2d1-4127-b18a-8d6e35ccd821 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:41.190377 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ded3a-b2d1-4127-b18a-8d6e35ccd821 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:41.190653 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ded3a-b2d1-4127-b18a-8d6e35ccd821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:41.199326 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:41.207838 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-777ded3a-b2d1-4127-b18a-8d6e35ccd821 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:41.209069 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-777ded3a-b2d1-4127-b18a-8d6e35ccd821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:41.249795 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ecbd14-3c86-4814-b083-0632ac2becaa None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:41.285005 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 253/499] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:41.285005 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ded3a-b2d1-4127-b18a-8d6e35ccd821 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:41.300653 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0cbfff-3a52-4999-b5ed-82e5b6991a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:41.301831 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0cbfff-3a52-4999-b5ed-82e5b6991a0f None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:41.302265 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0cbfff-3a52-4999-b5ed-82e5b6991a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:41.302682 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0cbfff-3a52-4999-b5ed-82e5b6991a0f None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:41.303822 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0cbfff-3a52-4999-b5ed-82e5b6991a0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:41.304309 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0cbfff-3a52-4999-b5ed-82e5b6991a0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:41.310905 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0cbfff-3a52-4999-b5ed-82e5b6991a0f None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:41.326099 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 247/500] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:41.326982 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0cbfff-3a52-4999-b5ed-82e5b6991a0f None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:41.335709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:41.409938 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ded3a-b2d1-4127-b18a-8d6e35ccd821 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:41.437896 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0cbfff-3a52-4999-b5ed-82e5b6991a0f None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:41.448827 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66110740-c4d7-4c5a-8692-220fb9bbf72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:41.449673 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66110740-c4d7-4c5a-8692-220fb9bbf72f None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:41.449956 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66110740-c4d7-4c5a-8692-220fb9bbf72f None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:41.450304 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66110740-c4d7-4c5a-8692-220fb9bbf72f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:41.455861 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:41.462715 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66110740-c4d7-4c5a-8692-220fb9bbf72f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:41.464393 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66110740-c4d7-4c5a-8692-220fb9bbf72f None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:41.477496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5fe12-6968-45c9-9121-3e0bf2b9d087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:41.478151 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5fe12-6968-45c9-9121-3e0bf2b9d087 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:41.478430 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5fe12-6968-45c9-9121-3e0bf2b9d087 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:41.478703 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc5fe12-6968-45c9-9121-3e0bf2b9d087 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:41.482441 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-66110740-c4d7-4c5a-8692-220fb9bbf72f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:41.482441 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:41.482441 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:41.482441 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:41.482441 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:41.482441 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:41.482441 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:41.482441 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:41.482441 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:41.482441 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:41.482441 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:41.482441 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:41.482441 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:09:41.482441 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:09:41.482441 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:09:41.483605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:09:41.483605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:41.483605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:41.483605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:41.483605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:41.483605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:41.483605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:41.483605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:41.483605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:41.483605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:41.483605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:41.483605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:41.483605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:41.483605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:41.483605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:41.483605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:41.483605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:41.484426 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:09:41.484426 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:09:41.484426 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:41.484426 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:41.488455 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 254/501] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:41.489347 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66110740-c4d7-4c5a-8692-220fb9bbf72f None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:41.549972 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5fe12-6968-45c9-9121-3e0bf2b9d087 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:41.551534 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc5fe12-6968-45c9-9121-3e0bf2b9d087 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:41.571936 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:41.589939 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66110740-c4d7-4c5a-8692-220fb9bbf72f None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:41.623426 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a9f09f-c4fb-4ddc-bd93-99260840cdc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:41.624221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9f09f-c4fb-4ddc-bd93-99260840cdc0 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:41.624503 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9f09f-c4fb-4ddc-bd93-99260840cdc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:41.624776 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9f09f-c4fb-4ddc-bd93-99260840cdc0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:41.634871 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a9f09f-c4fb-4ddc-bd93-99260840cdc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:41.642497 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-29a9f09f-c4fb-4ddc-bd93-99260840cdc0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:09:41.642497 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:41.642497 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:41.642497 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:41.642497 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:41.642497 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:41.642497 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:41.642497 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:41.642497 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:41.642497 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:41.642497 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:41.642497 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:41.642497 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:41.642497 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:41.642497 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:41.642497 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:41.643454 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:41.643454 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:41.643454 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:41.643454 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:41.643454 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:41.643454 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:41.643454 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:41.643454 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:41.643454 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:41.643454 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:41.643454 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:41.643454 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:41.643454 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:41.643454 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:41.643454 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:41.643454 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:41.644247 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:41.644247 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:41.644247 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:41.644247 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:41.644247 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:41.644247 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:41.644247 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:41.644247 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:41.644247 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:41.644247 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:41.644247 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:09:41.644247 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:41.648802 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 255/502] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 03:09:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:41.649470 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a9f09f-c4fb-4ddc-bd93-99260840cdc0 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:41.655147 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 248/503] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:09:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:09:41.655979 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5fe12-6968-45c9-9121-3e0bf2b9d087 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:41.673816 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:41.749584 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a9f09f-c4fb-4ddc-bd93-99260840cdc0 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:41.754827 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc5fe12-6968-45c9-9121-3e0bf2b9d087 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:41.784050 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5024f226-d991-4b02-b63d-afffe86a9534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:41.785235 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5024f226-d991-4b02-b63d-afffe86a9534 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:41.785515 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5024f226-d991-4b02-b63d-afffe86a9534 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:41.785785 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5024f226-d991-4b02-b63d-afffe86a9534 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:41.788567 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff3b12-ba41-43e9-99b4-872baf080af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:41.789260 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff3b12-ba41-43e9-99b4-872baf080af6 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:41.789541 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff3b12-ba41-43e9-99b4-872baf080af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:41.789816 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff3b12-ba41-43e9-99b4-872baf080af6 None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:41.790550 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff3b12-ba41-43e9-99b4-872baf080af6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:41.790844 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff3b12-ba41-43e9-99b4-872baf080af6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:41.792239 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff3b12-ba41-43e9-99b4-872baf080af6 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:41.793839 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5024f226-d991-4b02-b63d-afffe86a9534 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:41.795287 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5024f226-d991-4b02-b63d-afffe86a9534 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:41.803766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 249/504] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:41.804939 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b036535-df42-480d-9bba-49270c3f5abb None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:41.805256 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b036535-df42-480d-9bba-49270c3f5abb None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:41.805537 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b036535-df42-480d-9bba-49270c3f5abb None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:41.806754 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 250/505] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:41.808835 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b036535-df42-480d-9bba-49270c3f5abb None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:41.811275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-5024f226-d991-4b02-b63d-afffe86a9534 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 27 03:09:41.811275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:41.811275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:41.811275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:41.811275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:41.811275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:41.811275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:41.811275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:41.811275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:41.811275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:41.811275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:41.811275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:41.811275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:09:41.811275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:09:41.811275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:09:41.811275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:09:41.812334 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:41.812334 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:41.812334 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:09:41.812334 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:09:41.812334 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:09:41.812334 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:09:41.812334 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:41.812334 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:41.812334 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:41.812334 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:41.812334 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:41.812334 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:41.812334 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:41.812334 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:41.812334 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:41.812334 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:41.812334 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:41.812334 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:41.813790 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:41.813790 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:41.813790 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:09:41.813790 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:09:41.813790 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:09:41.813790 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:09:41.813790 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:09:41.813790 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:09:41.813790 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 27 03:09:41.813790 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:41.813790 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 256/506] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:09:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:41.813790 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5024f226-d991-4b02-b63d-afffe86a9534 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:41.816908 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:41.888982 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:41.913368 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b036535-df42-480d-9bba-49270c3f5abb None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:41.915017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5024f226-d991-4b02-b63d-afffe86a9534 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:41.948449 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b712fbd-1dc4-4ced-8083-2df3279c3072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:41.949248 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b712fbd-1dc4-4ced-8083-2df3279c3072 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:41.949578 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b712fbd-1dc4-4ced-8083-2df3279c3072 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:41.949921 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b712fbd-1dc4-4ced-8083-2df3279c3072 None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:41.950559 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65cd1de-64ec-497a-a567-97ed844e454e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:41.950797 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b712fbd-1dc4-4ced-8083-2df3279c3072 None admin] RBAC: Adding query filter params (name=service) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:41.951166 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b712fbd-1dc4-4ced-8083-2df3279c3072 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:41.951356 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65cd1de-64ec-497a-a567-97ed844e454e None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:41.951632 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65cd1de-64ec-497a-a567-97ed844e454e None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:41.951898 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65cd1de-64ec-497a-a567-97ed844e454e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:41.952797 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b712fbd-1dc4-4ced-8083-2df3279c3072 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:41.961958 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 251/507] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 03:09:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:41.962752 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b712fbd-1dc4-4ced-8083-2df3279c3072 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:41.963231 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65cd1de-64ec-497a-a567-97ed844e454e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:41.964691 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65cd1de-64ec-497a-a567-97ed844e454e None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:41.982664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-f65cd1de-64ec-497a-a567-97ed844e454e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:41.982664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:41.982664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:41.982664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:41.982664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:41.982664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:41.982664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:41.982664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:41.982664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:41.982664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:41.982664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:41.982664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:41.982664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:09:41.982664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:09:41.982664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:09:41.983536 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:09:41.983536 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:41.983536 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:41.983536 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:41.983536 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:41.983536 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:41.983536 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:41.983536 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:41.983536 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:41.983536 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:41.983536 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:41.983536 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:41.983536 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:41.983536 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:41.983536 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:41.983536 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:41.983536 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:41.984372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:09:41.984372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:09:41.984372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:41.984372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:41.984372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 257/508] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:41.985034 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22265e0-06c2-46fa-b99a-1babe4dbe432 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:41.985312 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22265e0-06c2-46fa-b99a-1babe4dbe432 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:41.985581 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22265e0-06c2-46fa-b99a-1babe4dbe432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:42.001819 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-b22265e0-06c2-46fa-b99a-1babe4dbe432 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:42.002991 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-b22265e0-06c2-46fa-b99a-1babe4dbe432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:42.053258 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:42.068334 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b712fbd-1dc4-4ced-8083-2df3279c3072 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:42.071690 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 258/509] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:42.072322 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22265e0-06c2-46fa-b99a-1babe4dbe432 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:42.104017 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d71119-0f20-4faf-93da-35b719fbec29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:42.104814 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d71119-0f20-4faf-93da-35b719fbec29 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:42.105166 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d71119-0f20-4faf-93da-35b719fbec29 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:42.105424 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d71119-0f20-4faf-93da-35b719fbec29 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:42.113208 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d71119-0f20-4faf-93da-35b719fbec29 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:42.114797 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d71119-0f20-4faf-93da-35b719fbec29 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:42.116521 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:42.129671 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-f2d71119-0f20-4faf-93da-35b719fbec29 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 27 03:09:42.129671 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:42.129671 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:42.129671 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:42.129671 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:42.129671 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:42.129671 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:42.129671 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:42.129671 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:42.129671 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:42.129671 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:42.129671 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:42.129671 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:09:42.129671 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:09:42.129671 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:09:42.129671 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:09:42.130911 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:42.130911 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:42.130911 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:09:42.130911 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:09:42.130911 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:09:42.130911 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:09:42.130911 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:42.130911 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:42.130911 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:42.130911 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:42.130911 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:42.130911 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:42.130911 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:42.130911 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:42.130911 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:42.130911 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:42.130911 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:42.130911 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:42.132095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:42.132095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:42.132095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:09:42.132095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:09:42.132095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:09:42.132095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:09:42.132095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:09:42.132095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:09:42.132095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 27 03:09:42.132095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:42.132095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 252/510] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:09:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:42.132095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d71119-0f20-4faf-93da-35b719fbec29 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:42.193294 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22265e0-06c2-46fa-b99a-1babe4dbe432 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:42.228034 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f371aed-83d2-4efe-97ec-7e07d03d89a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:42.228709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f371aed-83d2-4efe-97ec-7e07d03d89a5 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:42.228987 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f371aed-83d2-4efe-97ec-7e07d03d89a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:42.229282 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f371aed-83d2-4efe-97ec-7e07d03d89a5 None admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:42.230114 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f371aed-83d2-4efe-97ec-7e07d03d89a5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:42.230418 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f371aed-83d2-4efe-97ec-7e07d03d89a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:42.230510 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d71119-0f20-4faf-93da-35b719fbec29 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:42.231883 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f371aed-83d2-4efe-97ec-7e07d03d89a5 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:42.250598 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 259/511] 10.222.0.50 () {56 vars in 1137 bytes} [Fri May 27 03:09:42 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:42.251418 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f371aed-83d2-4efe-97ec-7e07d03d89a5 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:42.265968 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d61da-9d92-42b6-85f5-2300a4c69c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:42.266773 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d61da-9d92-42b6-85f5-2300a4c69c52 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:42.267062 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d61da-9d92-42b6-85f5-2300a4c69c52 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:42.267334 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d61da-9d92-42b6-85f5-2300a4c69c52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:42.278132 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4d61da-9d92-42b6-85f5-2300a4c69c52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:42.279617 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4d61da-9d92-42b6-85f5-2300a4c69c52 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:42.297750 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-aa4d61da-9d92-42b6-85f5-2300a4c69c52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:42.297750 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:42.297750 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:42.297750 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:42.297750 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:42.297750 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:42.297750 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:42.297750 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:42.297750 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:42.297750 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:42.297750 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:42.297750 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:42.297750 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:09:42.297750 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:09:42.297750 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:09:42.298699 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:09:42.298699 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:42.298699 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:42.298699 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:42.298699 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:42.298699 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:42.298699 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:42.298699 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:42.298699 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:42.298699 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:42.298699 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:42.298699 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:42.298699 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:42.298699 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:42.298699 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:42.298699 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:42.298699 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:42.299672 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:09:42.299672 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:09:42.299672 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:42.299672 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:42.299672 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 253/512] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:42.299672 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d61da-9d92-42b6-85f5-2300a4c69c52 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:42.305411 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:42.352372 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:42.362916 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f371aed-83d2-4efe-97ec-7e07d03d89a5 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:42.399497 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52361e5-096d-475f-8cff-059960edc0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:42.400169 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52361e5-096d-475f-8cff-059960edc0e6 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:42.400452 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52361e5-096d-475f-8cff-059960edc0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:42.400749 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52361e5-096d-475f-8cff-059960edc0e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:42.401498 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52361e5-096d-475f-8cff-059960edc0e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:42.401793 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52361e5-096d-475f-8cff-059960edc0e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:42.403185 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52361e5-096d-475f-8cff-059960edc0e6 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:42.404558 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d61da-9d92-42b6-85f5-2300a4c69c52 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:42.415607 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 260/513] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:09:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:42.416240 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52361e5-096d-475f-8cff-059960edc0e6 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:42.444125 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71059212-34d1-48ac-8d69-a2ef7a474c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:42.445169 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71059212-34d1-48ac-8d69-a2ef7a474c56 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:42.445449 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71059212-34d1-48ac-8d69-a2ef7a474c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:42.445783 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71059212-34d1-48ac-8d69-a2ef7a474c56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:42.459204 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71059212-34d1-48ac-8d69-a2ef7a474c56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:42.460733 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71059212-34d1-48ac-8d69-a2ef7a474c56 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:42.478681 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-71059212-34d1-48ac-8d69-a2ef7a474c56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:42.478681 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:42.478681 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:42.478681 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:42.478681 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:42.478681 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:42.478681 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:42.478681 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:42.478681 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:42.478681 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:42.478681 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:42.478681 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:42.478681 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:09:42.478681 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:09:42.478681 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:09:42.479546 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:09:42.479546 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:42.479546 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:42.479546 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:42.479546 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:42.479546 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:42.479546 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:42.479546 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:42.479546 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:42.479546 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:42.479546 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:42.479546 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:42.479546 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:42.479546 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:42.479546 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:42.479546 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:42.479546 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:42.480394 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:09:42.480394 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:09:42.480394 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:42.480394 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:42.480394 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 254/514] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:42.480666 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71059212-34d1-48ac-8d69-a2ef7a474c56 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:42.522902 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52361e5-096d-475f-8cff-059960edc0e6 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:42.524467 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:42.557750 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d9045-1853-48e0-85df-72ca341d2ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:42.558419 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d9045-1853-48e0-85df-72ca341d2ae0 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:42.558696 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d9045-1853-48e0-85df-72ca341d2ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:42.558964 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d9045-1853-48e0-85df-72ca341d2ae0 None admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:42.559793 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d9045-1853-48e0-85df-72ca341d2ae0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:42.560101 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d9045-1853-48e0-85df-72ca341d2ae0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:42.561575 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d9045-1853-48e0-85df-72ca341d2ae0 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:42.578959 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 261/515] 10.222.0.50 () {56 vars in 1137 bytes} [Fri May 27 03:09:42 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:42.579794 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d9045-1853-48e0-85df-72ca341d2ae0 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:42.580908 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71059212-34d1-48ac-8d69-a2ef7a474c56 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:42.582482 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:42.617453 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a957a55-ea06-4f4c-879e-a034a7a61fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:42.618023 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a957a55-ea06-4f4c-879e-a034a7a61fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:42.618305 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a957a55-ea06-4f4c-879e-a034a7a61fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:42.618578 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a957a55-ea06-4f4c-879e-a034a7a61fa3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:42.681650 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d9045-1853-48e0-85df-72ca341d2ae0 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:42.686347 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a957a55-ea06-4f4c-879e-a034a7a61fa3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:42.687898 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a957a55-ea06-4f4c-879e-a034a7a61fa3 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:42.717455 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab955d03-4e3a-4a84-a38d-7acfbda5d8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:42.718126 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab955d03-4e3a-4a84-a38d-7acfbda5d8b4 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:42.718407 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab955d03-4e3a-4a84-a38d-7acfbda5d8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:42.718683 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab955d03-4e3a-4a84-a38d-7acfbda5d8b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:42.719403 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab955d03-4e3a-4a84-a38d-7acfbda5d8b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:42.719707 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab955d03-4e3a-4a84-a38d-7acfbda5d8b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:42.721124 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab955d03-4e3a-4a84-a38d-7acfbda5d8b4 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:42.732695 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 262/516] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:09:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:42.733548 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab955d03-4e3a-4a84-a38d-7acfbda5d8b4 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:42.755349 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:42.790474 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 255/517] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:09:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:09:42.791316 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a957a55-ea06-4f4c-879e-a034a7a61fa3 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:42.793663 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:42.834941 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab955d03-4e3a-4a84-a38d-7acfbda5d8b4 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:42.868414 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5add1fa-4c6a-47b8-976f-58645a1b6410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:42.869188 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5add1fa-4c6a-47b8-976f-58645a1b6410 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:42.869448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5add1fa-4c6a-47b8-976f-58645a1b6410 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:42.869720 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5add1fa-4c6a-47b8-976f-58645a1b6410 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:42.876707 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5add1fa-4c6a-47b8-976f-58645a1b6410 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:42.878218 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5add1fa-4c6a-47b8-976f-58645a1b6410 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:42.890562 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a957a55-ea06-4f4c-879e-a034a7a61fa3 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:42.893982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-f5add1fa-4c6a-47b8-976f-58645a1b6410 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:09:42.893982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:42.893982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:42.893982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:42.893982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:42.893982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:42.893982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:42.893982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:42.893982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:42.893982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:42.893982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:42.893982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:42.893982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:09:42.893982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:09:42.893982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:09:42.894890 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:09:42.894890 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:42.894890 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:42.894890 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:42.894890 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:42.894890 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:42.894890 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:42.894890 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:42.894890 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:42.894890 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:42.894890 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:42.894890 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:42.894890 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:42.894890 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:42.894890 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:42.894890 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:42.894890 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:42.895796 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:09:42.895796 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:09:42.895796 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:09:42.895796 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:09:42.895796 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:09:42.895796 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:09:42.895796 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:09:42.895796 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:42.895796 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 263/518] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:09:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:42.896253 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5add1fa-4c6a-47b8-976f-58645a1b6410 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:42.927574 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e0b3ab-452b-471a-bbc1-16cf91b68a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:42.928241 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0b3ab-452b-471a-bbc1-16cf91b68a39 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:42.928527 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0b3ab-452b-471a-bbc1-16cf91b68a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:42.928802 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e0b3ab-452b-471a-bbc1-16cf91b68a39 None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:42.929556 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e0b3ab-452b-471a-bbc1-16cf91b68a39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:42.929868 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e0b3ab-452b-471a-bbc1-16cf91b68a39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:42.931252 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e0b3ab-452b-471a-bbc1-16cf91b68a39 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:42.942682 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 256/519] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:09:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:42.943651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e0b3ab-452b-471a-bbc1-16cf91b68a39 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:42.987704 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:43.005556 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:43.007205 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5add1fa-4c6a-47b8-976f-58645a1b6410 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:43.043896 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e78c0-ed24-4811-9748-22dc65be8c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:43.044557 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e78c0-ed24-4811-9748-22dc65be8c3a None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:43.044854 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e78c0-ed24-4811-9748-22dc65be8c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:43.045155 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2e78c0-ed24-4811-9748-22dc65be8c3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:43.045951 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e0b3ab-452b-471a-bbc1-16cf91b68a39 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:43.080366 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a179720-d889-4bc1-93c7-a8d85385db74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:43.081202 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a179720-d889-4bc1-93c7-a8d85385db74 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:43.081481 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a179720-d889-4bc1-93c7-a8d85385db74 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:43.081753 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a179720-d889-4bc1-93c7-a8d85385db74 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:43.089298 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a179720-d889-4bc1-93c7-a8d85385db74 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:43.090754 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a179720-d889-4bc1-93c7-a8d85385db74 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:43.104923 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-0a179720-d889-4bc1-93c7-a8d85385db74 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 27 03:09:43.104923 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:43.104923 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:43.104923 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:43.104923 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:43.104923 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:43.104923 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:43.104923 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:43.104923 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:43.104923 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:43.104923 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:43.104923 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:43.104923 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:09:43.104923 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:09:43.104923 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:09:43.104923 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:09:43.105823 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:43.105823 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:43.105823 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:09:43.105823 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:09:43.105823 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:09:43.105823 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:09:43.105823 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:43.105823 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:43.105823 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:43.105823 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:43.105823 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:43.105823 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:43.105823 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:43.105823 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:43.105823 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:43.105823 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:43.105823 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:43.105823 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:43.106704 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:43.106704 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:43.106704 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:09:43.106704 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:09:43.106704 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:09:43.106704 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:09:43.106704 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:09:43.106704 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:09:43.106704 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 27 03:09:43.106704 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:43.106704 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 257/520] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 03:09:42 2022] GET /identity/v3/users/neutron => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:43.107237 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a179720-d889-4bc1-93c7-a8d85385db74 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:43.117622 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e78c0-ed24-4811-9748-22dc65be8c3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:43.119162 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2e78c0-ed24-4811-9748-22dc65be8c3a None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:43.198750 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:43.214720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a179720-d889-4bc1-93c7-a8d85385db74 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:43.218164 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 264/521] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:09:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:09:43.218981 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e78c0-ed24-4811-9748-22dc65be8c3a None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:43.239044 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:43.252590 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b176b0-6386-49ab-93e0-4db9a509cf0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:43.253445 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b176b0-6386-49ab-93e0-4db9a509cf0f None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:43.253720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b176b0-6386-49ab-93e0-4db9a509cf0f None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:43.253989 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b176b0-6386-49ab-93e0-4db9a509cf0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:43.265347 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b176b0-6386-49ab-93e0-4db9a509cf0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:43.266881 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b176b0-6386-49ab-93e0-4db9a509cf0f None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:43.283943 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-f1b176b0-6386-49ab-93e0-4db9a509cf0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:43.283943 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:43.283943 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:43.283943 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:43.283943 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:43.283943 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:43.283943 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:43.283943 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:43.283943 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:43.283943 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:43.283943 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:43.283943 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:43.283943 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:09:43.283943 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:09:43.283943 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:09:43.285205 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:09:43.285205 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:43.285205 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:43.285205 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:43.285205 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:43.285205 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:43.285205 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:43.285205 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:43.285205 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:43.285205 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:43.285205 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:43.285205 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:43.285205 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:43.285205 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:43.285205 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:43.285205 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:43.285205 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:43.286107 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:09:43.286107 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:09:43.286107 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:43.286107 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:43.286107 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 258/522] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:43.286521 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b176b0-6386-49ab-93e0-4db9a509cf0f None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:43.327138 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2e78c0-ed24-4811-9748-22dc65be8c3a None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:43.362034 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c511a-423a-4154-b5d4-3e5e0b853683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:43.362816 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c511a-423a-4154-b5d4-3e5e0b853683 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:43.363090 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c511a-423a-4154-b5d4-3e5e0b853683 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:43.363363 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c511a-423a-4154-b5d4-3e5e0b853683 None admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:43.364360 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9c511a-423a-4154-b5d4-3e5e0b853683 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:43.364728 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9c511a-423a-4154-b5d4-3e5e0b853683 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:43.366241 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9c511a-423a-4154-b5d4-3e5e0b853683 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:43.384078 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 265/523] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 03:09:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:43.384923 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c511a-423a-4154-b5d4-3e5e0b853683 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:43.387390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b176b0-6386-49ab-93e0-4db9a509cf0f None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:43.422310 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cfcfcf-c58b-4223-994c-57eaf964a917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:43.422982 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cfcfcf-c58b-4223-994c-57eaf964a917 None admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:43.423262 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cfcfcf-c58b-4223-994c-57eaf964a917 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:43.423533 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cfcfcf-c58b-4223-994c-57eaf964a917 None admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:43.424234 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cfcfcf-c58b-4223-994c-57eaf964a917 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:43.425711 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cfcfcf-c58b-4223-994c-57eaf964a917 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:43.428415 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:43.446851 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 259/524] 10.222.0.50 () {60 vars in 1132 bytes} [Fri May 27 03:09:43 2022] POST /identity/v3/users => generated 247 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:09:43.447660 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cfcfcf-c58b-4223-994c-57eaf964a917 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:43.486445 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:43.504800 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c511a-423a-4154-b5d4-3e5e0b853683 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:43.544217 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ef400-6c49-4a0d-8303-13587cd469a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:43.544882 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ef400-6c49-4a0d-8303-13587cd469a6 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:43.545172 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ef400-6c49-4a0d-8303-13587cd469a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:43.545451 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834ef400-6c49-4a0d-8303-13587cd469a6 None admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:43.546288 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834ef400-6c49-4a0d-8303-13587cd469a6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:43.546590 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834ef400-6c49-4a0d-8303-13587cd469a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:43.548074 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834ef400-6c49-4a0d-8303-13587cd469a6 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:43.564705 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cfcfcf-c58b-4223-994c-57eaf964a917 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:43.567862 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 266/525] 10.222.0.50 () {56 vars in 1139 bytes} [Fri May 27 03:09:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:43.573455 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ef400-6c49-4a0d-8303-13587cd469a6 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:43.605526 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dccadf-c416-4e21-afdd-c35f8e732f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:43.606357 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dccadf-c416-4e21-afdd-c35f8e732f09 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:43.606639 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dccadf-c416-4e21-afdd-c35f8e732f09 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:43.606914 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dccadf-c416-4e21-afdd-c35f8e732f09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:43.619445 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dccadf-c416-4e21-afdd-c35f8e732f09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:43.620937 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dccadf-c416-4e21-afdd-c35f8e732f09 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:43.639379 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-f9dccadf-c416-4e21-afdd-c35f8e732f09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:43.639379 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:43.639379 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:43.639379 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:43.639379 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:43.639379 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:43.639379 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:43.639379 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:43.639379 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:43.639379 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:43.639379 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:43.639379 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:43.639379 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:09:43.639379 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:09:43.639379 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:09:43.641032 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:09:43.641032 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:43.641032 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:43.641032 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:43.641032 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:43.641032 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:43.641032 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:43.641032 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:43.641032 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:43.641032 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:43.641032 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:43.641032 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:43.641032 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:43.641032 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:43.641032 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:43.641032 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:43.641032 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:43.641937 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:09:43.641937 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:09:43.641937 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:43.641937 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:43.641937 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 260/526] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:43.641937 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dccadf-c416-4e21-afdd-c35f8e732f09 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:43.697080 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:43.701505 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834ef400-6c49-4a0d-8303-13587cd469a6 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:43.738478 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370c7a41-afc9-4e3b-a5bd-24411eda60b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:43.739253 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c7a41-afc9-4e3b-a5bd-24411eda60b0 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:43.739518 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c7a41-afc9-4e3b-a5bd-24411eda60b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:43.739763 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370c7a41-afc9-4e3b-a5bd-24411eda60b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:43.740669 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370c7a41-afc9-4e3b-a5bd-24411eda60b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:43.741200 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370c7a41-afc9-4e3b-a5bd-24411eda60b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:43.741743 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:43.742407 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370c7a41-afc9-4e3b-a5bd-24411eda60b0 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:43.754928 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 267/527] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:09:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:43.755502 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370c7a41-afc9-4e3b-a5bd-24411eda60b0 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:43.765950 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dccadf-c416-4e21-afdd-c35f8e732f09 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:43.801800 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec3a39a-4c4c-4e43-a0a1-5d25ac518090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:43.802460 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec3a39a-4c4c-4e43-a0a1-5d25ac518090 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:43.802744 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec3a39a-4c4c-4e43-a0a1-5d25ac518090 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:43.803023 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec3a39a-4c4c-4e43-a0a1-5d25ac518090 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:43.803863 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec3a39a-4c4c-4e43-a0a1-5d25ac518090 None admin] RBAC: Adding query filter params (role.id=3451aa51e64642768b2228126d20fa03, scope.project.id=f8398941252346468483ea0c863122f4, user.id=1a699988f08a4001b3b7d764f01f4fc5) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:43.804174 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec3a39a-4c4c-4e43-a0a1-5d25ac518090 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:43.805677 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec3a39a-4c4c-4e43-a0a1-5d25ac518090 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:43.811627 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 261/528] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:09:43 2022] GET /identity/v3/role_assignments?user.id=1a699988f08a4001b3b7d764f01f4fc5&scope.project.id=f8398941252346468483ea0c863122f4&role.id=3451aa51e64642768b2228126d20fa03 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:43.812244 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec3a39a-4c4c-4e43-a0a1-5d25ac518090 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:43.881463 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370c7a41-afc9-4e3b-a5bd-24411eda60b0 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:43.920730 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01042074-b4df-4c84-8652-a55b317d9be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:43.921505 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01042074-b4df-4c84-8652-a55b317d9be7 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:43.921793 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01042074-b4df-4c84-8652-a55b317d9be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:43.922070 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01042074-b4df-4c84-8652-a55b317d9be7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:43.956333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:43.960299 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec3a39a-4c4c-4e43-a0a1-5d25ac518090 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:44.005203 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01042074-b4df-4c84-8652-a55b317d9be7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:44.006442 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ad870-b41b-4bc6-ab24-a3e5ea8a5dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:44.006909 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01042074-b4df-4c84-8652-a55b317d9be7 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:44.007126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ad870-b41b-4bc6-ab24-a3e5ea8a5dba None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:44.007406 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ad870-b41b-4bc6-ab24-a3e5ea8a5dba None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:44.007676 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ad870-b41b-4bc6-ab24-a3e5ea8a5dba None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:44.008405 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ad870-b41b-4bc6-ab24-a3e5ea8a5dba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:44.009777 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ad870-b41b-4bc6-ab24-a3e5ea8a5dba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:44.011166 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ad870-b41b-4bc6-ab24-a3e5ea8a5dba None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:44.026454 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 262/529] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:09:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:44.027216 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ad870-b41b-4bc6-ab24-a3e5ea8a5dba None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:44.073346 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:44.124661 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 268/530] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:09:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:09:44.125960 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01042074-b4df-4c84-8652-a55b317d9be7 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:44.134298 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ad870-b41b-4bc6-ab24-a3e5ea8a5dba None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:44.169864 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413722d9-44a2-4172-a7ef-f19fb05bb820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:44.170435 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413722d9-44a2-4172-a7ef-f19fb05bb820 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:44.170706 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413722d9-44a2-4172-a7ef-f19fb05bb820 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:44.171022 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413722d9-44a2-4172-a7ef-f19fb05bb820 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:44.178461 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413722d9-44a2-4172-a7ef-f19fb05bb820 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:44.179972 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413722d9-44a2-4172-a7ef-f19fb05bb820 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:44.188786 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:44.194856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-413722d9-44a2-4172-a7ef-f19fb05bb820 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:09:44.194856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:44.194856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:44.194856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:44.194856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:44.194856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:44.194856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:44.194856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:44.194856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:44.194856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:44.194856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:44.194856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:44.194856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:09:44.194856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:09:44.194856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:09:44.195720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:09:44.195720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:44.195720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:44.195720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:44.195720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:44.195720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:44.195720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:44.195720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:44.195720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:44.195720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:44.195720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:44.195720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:44.195720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:44.195720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:44.195720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:44.195720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:44.195720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:44.196558 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:09:44.196558 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:09:44.196558 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:09:44.196558 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:09:44.196558 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:09:44.196558 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:09:44.196558 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:09:44.196558 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:44.201018 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 263/531] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:09:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:44.201918 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413722d9-44a2-4172-a7ef-f19fb05bb820 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:44.236666 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01042074-b4df-4c84-8652-a55b317d9be7 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:44.273669 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ba887f-604a-4018-8909-4ea59210750d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:44.274868 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba887f-604a-4018-8909-4ea59210750d None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:44.275346 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba887f-604a-4018-8909-4ea59210750d None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:44.275827 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba887f-604a-4018-8909-4ea59210750d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:44.283323 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ba887f-604a-4018-8909-4ea59210750d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:44.285440 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ba887f-604a-4018-8909-4ea59210750d None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:44.300580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-13ba887f-604a-4018-8909-4ea59210750d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:09:44.300580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:44.300580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:44.300580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:44.300580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:44.300580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:44.300580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:44.300580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:44.300580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:44.300580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:44.300580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:44.300580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:44.300580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:09:44.300580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:09:44.300580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:09:44.301450 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:09:44.301450 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:44.301450 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:44.301450 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:44.301450 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:44.301450 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:44.301450 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:44.301450 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:44.301450 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:44.301450 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:44.301450 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:44.301450 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:44.301450 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:44.301450 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:44.301450 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:44.301450 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:44.301450 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:44.302305 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:09:44.302305 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:09:44.302305 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:09:44.302305 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:09:44.302305 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:09:44.302305 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:09:44.302305 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:09:44.302305 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:44.309621 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 269/532] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:09:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:44.310865 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ba887f-604a-4018-8909-4ea59210750d None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:44.311064 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:44.316591 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413722d9-44a2-4172-a7ef-f19fb05bb820 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:44.358151 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f9f3f2-a66e-4a08-84ad-6c34be7545d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:44.358960 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9f3f2-a66e-4a08-84ad-6c34be7545d5 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:44.359758 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9f3f2-a66e-4a08-84ad-6c34be7545d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:44.360045 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9f3f2-a66e-4a08-84ad-6c34be7545d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:44.375203 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f9f3f2-a66e-4a08-84ad-6c34be7545d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:44.377207 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f9f3f2-a66e-4a08-84ad-6c34be7545d5 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:44.399661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-38f9f3f2-a66e-4a08-84ad-6c34be7545d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:44.399661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:44.399661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:44.399661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:44.399661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:44.399661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:44.399661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:44.399661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:44.399661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:44.399661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:44.399661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:44.399661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:44.399661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:09:44.399661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:09:44.399661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:09:44.400550 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:09:44.400550 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:44.400550 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:44.400550 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:44.400550 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:44.400550 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:44.400550 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:44.400550 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:44.400550 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:44.400550 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:44.400550 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:44.400550 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:44.400550 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:44.400550 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:44.400550 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:44.400550 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:44.400550 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:44.401423 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:09:44.401423 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:09:44.401423 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:44.401423 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:44.407645 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 264/533] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:44.407645 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f9f3f2-a66e-4a08-84ad-6c34be7545d5 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:44.437181 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ba887f-604a-4018-8909-4ea59210750d None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:44.461489 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:44.485842 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94022095-0d9e-40b1-ba64-ce2462a0b27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:44.487409 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94022095-0d9e-40b1-ba64-ce2462a0b27a None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:44.487912 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94022095-0d9e-40b1-ba64-ce2462a0b27a None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:44.488381 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94022095-0d9e-40b1-ba64-ce2462a0b27a None admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:44.489832 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94022095-0d9e-40b1-ba64-ce2462a0b27a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:44.490173 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94022095-0d9e-40b1-ba64-ce2462a0b27a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:44.491642 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94022095-0d9e-40b1-ba64-ce2462a0b27a None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:44.509787 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 270/534] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 03:09:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:44.510751 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94022095-0d9e-40b1-ba64-ce2462a0b27a None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:44.523436 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f9f3f2-a66e-4a08-84ad-6c34be7545d5 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:44.546171 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:44.569615 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38dd325-8f12-4c12-9242-e3ab6e46c0e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:44.570836 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38dd325-8f12-4c12-9242-e3ab6e46c0e1 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:44.571627 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38dd325-8f12-4c12-9242-e3ab6e46c0e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:44.572358 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38dd325-8f12-4c12-9242-e3ab6e46c0e1 None admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:44.573552 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38dd325-8f12-4c12-9242-e3ab6e46c0e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:44.574358 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38dd325-8f12-4c12-9242-e3ab6e46c0e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:44.576524 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38dd325-8f12-4c12-9242-e3ab6e46c0e1 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:44.605398 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 265/535] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 03:09:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:44.609180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38dd325-8f12-4c12-9242-e3ab6e46c0e1 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:44.633528 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94022095-0d9e-40b1-ba64-ce2462a0b27a None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:44.687452 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc05b1f-55ed-4cd2-8ed7-8f3a52d0d71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:44.688491 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc05b1f-55ed-4cd2-8ed7-8f3a52d0d71d None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:44.689014 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc05b1f-55ed-4cd2-8ed7-8f3a52d0d71d None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:44.689485 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc05b1f-55ed-4cd2-8ed7-8f3a52d0d71d None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:44.690415 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc05b1f-55ed-4cd2-8ed7-8f3a52d0d71d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:44.691405 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc05b1f-55ed-4cd2-8ed7-8f3a52d0d71d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:44.693665 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc05b1f-55ed-4cd2-8ed7-8f3a52d0d71d None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:44.715891 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 271/536] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:09:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:44.715891 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc05b1f-55ed-4cd2-8ed7-8f3a52d0d71d None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:44.729379 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38dd325-8f12-4c12-9242-e3ab6e46c0e1 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:44.757913 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:44.777557 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6718f-dccc-41ad-b192-7a2430ca1f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:44.778857 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6718f-dccc-41ad-b192-7a2430ca1f41 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:44.779689 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6718f-dccc-41ad-b192-7a2430ca1f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:44.780433 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c6718f-dccc-41ad-b192-7a2430ca1f41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:44.781758 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6718f-dccc-41ad-b192-7a2430ca1f41 None admin] RBAC: Adding query filter params (role.id=3451aa51e64642768b2228126d20fa03, scope.project.id=f8398941252346468483ea0c863122f4, user.id=5b3c9c5c8d2e4376b1b70d974d4cb851) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:44.782611 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6718f-dccc-41ad-b192-7a2430ca1f41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:44.784843 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c6718f-dccc-41ad-b192-7a2430ca1f41 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:44.793462 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 266/537] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:09:44 2022] GET /identity/v3/role_assignments?user.id=5b3c9c5c8d2e4376b1b70d974d4cb851&scope.project.id=f8398941252346468483ea0c863122f4&role.id=3451aa51e64642768b2228126d20fa03 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:44.794726 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6718f-dccc-41ad-b192-7a2430ca1f41 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:44.852805 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:44.887334 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc05b1f-55ed-4cd2-8ed7-8f3a52d0d71d None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:44.925615 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c6718f-dccc-41ad-b192-7a2430ca1f41 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:44.969653 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f3347d-36dc-4781-a2b6-bf3c27c3e3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:44.971100 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3347d-36dc-4781-a2b6-bf3c27c3e3f5 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:44.971573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3347d-36dc-4781-a2b6-bf3c27c3e3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:44.972032 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3347d-36dc-4781-a2b6-bf3c27c3e3f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:44.981107 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f3347d-36dc-4781-a2b6-bf3c27c3e3f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:44.981699 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12700c03-92cd-4703-b524-2cdc2d4e792b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:44.982385 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12700c03-92cd-4703-b524-2cdc2d4e792b None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:44.982670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12700c03-92cd-4703-b524-2cdc2d4e792b None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:44.982950 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12700c03-92cd-4703-b524-2cdc2d4e792b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:44.983796 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12700c03-92cd-4703-b524-2cdc2d4e792b None admin] RBAC: Adding query filter params (role.id=3451aa51e64642768b2228126d20fa03, scope.project.id=f8398941252346468483ea0c863122f4, user.id=36dd9b94ceb74eb4a7e9b5bbd5e8d8ba) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:44.984047 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f3347d-36dc-4781-a2b6-bf3c27c3e3f5 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:44.984156 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12700c03-92cd-4703-b524-2cdc2d4e792b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:44.990562 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12700c03-92cd-4703-b524-2cdc2d4e792b None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:44.995806 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 272/538] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:09:44 2022] GET /identity/v3/role_assignments?user.id=36dd9b94ceb74eb4a7e9b5bbd5e8d8ba&scope.project.id=f8398941252346468483ea0c863122f4&role.id=3451aa51e64642768b2228126d20fa03 => generated 259 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:45.011119 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-05f3347d-36dc-4781-a2b6-bf3c27c3e3f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:09:45.011119 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:45.011119 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:45.011119 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:45.011119 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:45.011119 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:45.011119 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:45.011119 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:45.011119 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:45.011119 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:45.011119 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:45.011119 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:45.011119 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:09:45.011119 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:09:45.011119 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:09:45.012041 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:09:45.012041 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:45.012041 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:45.012041 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:45.012041 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:45.012041 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:45.012041 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:45.012041 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:45.012041 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:45.012041 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:45.012041 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:45.012041 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:45.012041 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:45.012041 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:45.012041 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:45.012041 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:45.012041 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:45.013007 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:09:45.013007 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:09:45.013007 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:09:45.013007 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:09:45.013007 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:09:45.013007 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:09:45.013007 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:09:45.013007 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:45.013007 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 267/539] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:09:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:45.019834 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12700c03-92cd-4703-b524-2cdc2d4e792b None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:45.178542 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12700c03-92cd-4703-b524-2cdc2d4e792b None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:45.180278 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:45.227107 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e728758-cb90-4b35-b008-951e7d1b78bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:45.227717 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e728758-cb90-4b35-b008-951e7d1b78bd None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:45.228005 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e728758-cb90-4b35-b008-951e7d1b78bd None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:45.228281 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e728758-cb90-4b35-b008-951e7d1b78bd None admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:45.229183 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e728758-cb90-4b35-b008-951e7d1b78bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:45.229495 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e728758-cb90-4b35-b008-951e7d1b78bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:45.230970 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e728758-cb90-4b35-b008-951e7d1b78bd None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:45.246535 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 273/540] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 03:09:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:45.252427 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f3347d-36dc-4781-a2b6-bf3c27c3e3f5 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:45.369325 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f3347d-36dc-4781-a2b6-bf3c27c3e3f5 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:45.371107 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:45.416641 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c73a50-e438-41ff-808b-8b85130e0d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:45.419711 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c73a50-e438-41ff-808b-8b85130e0d90 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:45.419711 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c73a50-e438-41ff-808b-8b85130e0d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:45.419711 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c73a50-e438-41ff-808b-8b85130e0d90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:45.419711 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c73a50-e438-41ff-808b-8b85130e0d90 None admin] RBAC: Adding query filter params (role.id=3451aa51e64642768b2228126d20fa03, scope.project.id=f8398941252346468483ea0c863122f4, user.id=4ee30c0d6d694ee8945800157a9fc5b7) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:45.419711 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c73a50-e438-41ff-808b-8b85130e0d90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:45.420434 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c73a50-e438-41ff-808b-8b85130e0d90 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:45.427198 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 268/541] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:09:45 2022] GET /identity/v3/role_assignments?user.id=4ee30c0d6d694ee8945800157a9fc5b7&scope.project.id=f8398941252346468483ea0c863122f4&role.id=3451aa51e64642768b2228126d20fa03 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:46.001458 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67801db3-820b-474c-91f8-db3b85e48361 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:46.002042 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67801db3-820b-474c-91f8-db3b85e48361 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:46.002042 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67801db3-820b-474c-91f8-db3b85e48361 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:46.003622 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 274/542] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:46.012566 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1e56a-4e1c-42f5-aeb5-9e568362fcf1 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:46.013076 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1e56a-4e1c-42f5-aeb5-9e568362fcf1 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:46.013272 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1e56a-4e1c-42f5-aeb5-9e568362fcf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:46.035382 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-61a1e56a-4e1c-42f5-aeb5-9e568362fcf1 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:46.036901 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-61a1e56a-4e1c-42f5-aeb5-9e568362fcf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:46.116865 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 269/543] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:46.152152 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67801db3-820b-474c-91f8-db3b85e48361 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:46.226544 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45bf0e6-1d68-449c-a6ee-cf23fa29e270 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:46.227199 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45bf0e6-1d68-449c-a6ee-cf23fa29e270 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:46.227664 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45bf0e6-1d68-449c-a6ee-cf23fa29e270 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:46.230618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 270/544] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:46.235036 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9369367-d23b-4e56-8342-1365192cd786 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:46.235540 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9369367-d23b-4e56-8342-1365192cd786 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:46.236006 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9369367-d23b-4e56-8342-1365192cd786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:46.253304 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-d9369367-d23b-4e56-8342-1365192cd786 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:46.254946 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-d9369367-d23b-4e56-8342-1365192cd786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:46.284221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67801db3-820b-474c-91f8-db3b85e48361 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:46.353201 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573567d0-fc97-40be-97fd-994ce75b7d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:46.354012 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573567d0-fc97-40be-97fd-994ce75b7d37 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:46.354288 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573567d0-fc97-40be-97fd-994ce75b7d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:46.354556 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573567d0-fc97-40be-97fd-994ce75b7d37 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:46.356907 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 271/545] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:09:46.368088 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573567d0-fc97-40be-97fd-994ce75b7d37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:46.375951 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-573567d0-fc97-40be-97fd-994ce75b7d37 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:09:46.375951 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:46.375951 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:46.375951 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:46.375951 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:46.375951 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:46.375951 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:46.375951 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:46.375951 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:46.375951 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:46.375951 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:46.375951 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:46.375951 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:46.375951 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:46.375951 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:46.375951 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:46.376809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:46.376809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:46.376809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:46.376809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:46.376809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:46.376809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:46.376809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:46.376809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:46.376809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:46.376809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:46.376809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:46.376809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:46.376809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:46.376809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:46.376809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:46.376809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:46.378024 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:46.378024 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:46.378024 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:46.378024 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:46.378024 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:46.378024 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:46.378024 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:46.378024 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:46.378024 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:46.378024 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:46.378024 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:09:46.378024 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:46.378024 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 275/546] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 03:09:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:46.383218 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9369367-d23b-4e56-8342-1365192cd786 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:46.404486 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573567d0-fc97-40be-97fd-994ce75b7d37 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:46.437327 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:46.450197 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:46.572110 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9369367-d23b-4e56-8342-1365192cd786 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:46.594632 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573567d0-fc97-40be-97fd-994ce75b7d37 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:46.637750 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84231fea-5d9f-44a7-a5e5-503b46281718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:46.638608 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84231fea-5d9f-44a7-a5e5-503b46281718 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:46.638882 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84231fea-5d9f-44a7-a5e5-503b46281718 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:46.639147 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84231fea-5d9f-44a7-a5e5-503b46281718 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:46.653402 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8084d49-8ded-45ae-891f-14c257c5b185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:46.654546 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8084d49-8ded-45ae-891f-14c257c5b185 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:46.654828 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8084d49-8ded-45ae-891f-14c257c5b185 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:46.655096 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8084d49-8ded-45ae-891f-14c257c5b185 None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:46.655851 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8084d49-8ded-45ae-891f-14c257c5b185 None admin] RBAC: Adding query filter params (name=service) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:46.656148 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8084d49-8ded-45ae-891f-14c257c5b185 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:46.657858 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8084d49-8ded-45ae-891f-14c257c5b185 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:46.660000 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84231fea-5d9f-44a7-a5e5-503b46281718 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:46.667074 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 272/547] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 03:09:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:46.668029 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-84231fea-5d9f-44a7-a5e5-503b46281718 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:09:46.668029 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:46.668029 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:46.668029 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:46.668029 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:46.668029 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:46.668029 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:46.668029 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:46.668029 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:46.668029 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:46.668029 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:46.668029 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:46.668029 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:46.668029 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:46.668029 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:46.668029 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:46.668893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:46.668893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:46.668893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:46.668893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:46.668893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:46.668893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:46.668893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:46.668893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:46.668893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:46.668893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:46.668893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:46.668893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:46.668893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:46.668893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:46.668893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:46.668893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:46.669785 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:46.669785 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:46.669785 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:46.669785 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:46.669785 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:46.669785 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:46.669785 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:46.669785 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:46.669785 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:46.669785 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:46.669785 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:09:46.669785 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:46.672532 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8084d49-8ded-45ae-891f-14c257c5b185 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:46.677067 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 276/548] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 03:09:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:09:46.682839 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84231fea-5d9f-44a7-a5e5-503b46281718 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:46.773545 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:46.787211 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8084d49-8ded-45ae-891f-14c257c5b185 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:46.826403 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fe11a8-d00b-41d8-a586-261c147f6091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:46.827120 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe11a8-d00b-41d8-a586-261c147f6091 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:46.827394 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe11a8-d00b-41d8-a586-261c147f6091 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:46.827661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe11a8-d00b-41d8-a586-261c147f6091 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:46.840669 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fe11a8-d00b-41d8-a586-261c147f6091 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:46.842871 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fe11a8-d00b-41d8-a586-261c147f6091 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:46.864161 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-e3fe11a8-d00b-41d8-a586-261c147f6091 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:46.864161 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:46.864161 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:46.864161 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:46.864161 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:46.864161 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:46.864161 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:46.864161 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:46.864161 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:46.864161 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:46.864161 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:46.864161 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:46.864161 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:09:46.864161 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:09:46.864161 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:09:46.865697 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:09:46.865697 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:46.865697 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:46.865697 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:46.865697 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:46.865697 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:46.865697 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:46.865697 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:46.865697 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:46.865697 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:46.865697 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:46.865697 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:46.865697 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:46.865697 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:46.865697 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:46.865697 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:46.865697 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:46.866681 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:09:46.866681 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:09:46.866681 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:46.866681 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:46.866681 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 273/549] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:09:46.870364 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fe11a8-d00b-41d8-a586-261c147f6091 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:46.887435 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:46.915996 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84231fea-5d9f-44a7-a5e5-503b46281718 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:46.974811 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5558e-a401-4890-bea4-97f651ef39db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:46.975849 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5558e-a401-4890-bea4-97f651ef39db None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:46.977499 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5558e-a401-4890-bea4-97f651ef39db None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:46.977974 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5558e-a401-4890-bea4-97f651ef39db None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:46.978914 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e5558e-a401-4890-bea4-97f651ef39db None admin] RBAC: Adding query filter params (name=service) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:46.980123 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e5558e-a401-4890-bea4-97f651ef39db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:46.982384 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e5558e-a401-4890-bea4-97f651ef39db None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:47.002420 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 277/550] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 03:09:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:47.006729 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5558e-a401-4890-bea4-97f651ef39db None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:47.064599 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fe11a8-d00b-41d8-a586-261c147f6091 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:47.130040 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b059afe-30bb-48af-b808-4356227ed8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:47.131079 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b059afe-30bb-48af-b808-4356227ed8d3 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:47.131565 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b059afe-30bb-48af-b808-4356227ed8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:47.132038 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b059afe-30bb-48af-b808-4356227ed8d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:47.132986 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b059afe-30bb-48af-b808-4356227ed8d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:47.133508 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b059afe-30bb-48af-b808-4356227ed8d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:47.135098 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b059afe-30bb-48af-b808-4356227ed8d3 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:47.153325 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:47.160204 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 274/551] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:09:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:47.166414 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b059afe-30bb-48af-b808-4356227ed8d3 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:47.178786 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5558e-a401-4890-bea4-97f651ef39db None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:47.213314 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a56bd8-260e-414e-8790-59e5185d5b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:47.214493 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a56bd8-260e-414e-8790-59e5185d5b08 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:47.214975 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a56bd8-260e-414e-8790-59e5185d5b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:47.215445 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a56bd8-260e-414e-8790-59e5185d5b08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:47.226904 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a56bd8-260e-414e-8790-59e5185d5b08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:47.228749 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a56bd8-260e-414e-8790-59e5185d5b08 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:47.238665 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:47.248382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-30a56bd8-260e-414e-8790-59e5185d5b08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:47.248382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:47.248382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:47.248382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:47.248382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:47.248382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:47.248382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:47.248382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:47.248382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:47.248382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:47.248382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:47.248382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:47.248382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:09:47.248382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:09:47.248382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:09:47.249636 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:09:47.249636 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:47.249636 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:47.249636 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:47.249636 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:47.249636 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:47.249636 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:47.249636 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:47.249636 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:47.249636 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:47.249636 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:47.249636 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:47.249636 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:47.249636 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:47.249636 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:47.249636 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:47.249636 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:47.250611 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:09:47.250611 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:09:47.250611 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:47.250611 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:47.255294 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 278/552] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:47.256574 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a56bd8-260e-414e-8790-59e5185d5b08 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:47.373312 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a56bd8-260e-414e-8790-59e5185d5b08 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:47.418872 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ac889-b88e-4d14-b134-14ab2c00ac4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:47.420078 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ac889-b88e-4d14-b134-14ab2c00ac4a None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:47.420591 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ac889-b88e-4d14-b134-14ab2c00ac4a None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:47.421063 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ac889-b88e-4d14-b134-14ab2c00ac4a None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:47.422056 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7ac889-b88e-4d14-b134-14ab2c00ac4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:47.422637 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7ac889-b88e-4d14-b134-14ab2c00ac4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:47.424582 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7ac889-b88e-4d14-b134-14ab2c00ac4a None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:47.440614 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 279/553] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:09:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:47.445668 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ac889-b88e-4d14-b134-14ab2c00ac4a None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:47.499128 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b059afe-30bb-48af-b808-4356227ed8d3 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:47.513163 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:47.563259 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ac889-b88e-4d14-b134-14ab2c00ac4a None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:47.598668 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37403c9-9abc-4d85-98dc-31c851e6b119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:47.600151 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37403c9-9abc-4d85-98dc-31c851e6b119 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:47.600605 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37403c9-9abc-4d85-98dc-31c851e6b119 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:47.600988 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37403c9-9abc-4d85-98dc-31c851e6b119 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:47.605203 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5337d050-af59-430a-b1d3-4e0ef1c8389c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:47.606039 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337d050-af59-430a-b1d3-4e0ef1c8389c None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:47.606326 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337d050-af59-430a-b1d3-4e0ef1c8389c None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:47.606591 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337d050-af59-430a-b1d3-4e0ef1c8389c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:47.613145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37403c9-9abc-4d85-98dc-31c851e6b119 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:47.615121 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37403c9-9abc-4d85-98dc-31c851e6b119 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:47.618310 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5337d050-af59-430a-b1d3-4e0ef1c8389c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:47.620769 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5337d050-af59-430a-b1d3-4e0ef1c8389c None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:47.633375 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-a37403c9-9abc-4d85-98dc-31c851e6b119 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 03:09:47.633375 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:47.633375 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:47.633375 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:47.633375 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:47.633375 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:47.633375 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:47.633375 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:47.633375 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:47.633375 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:47.633375 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:47.633375 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:47.633375 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:09:47.633375 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:09:47.633375 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:09:47.633375 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:09:47.634430 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:47.634430 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:47.634430 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:09:47.634430 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:09:47.634430 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:09:47.634430 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:09:47.634430 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:47.634430 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:47.634430 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:47.634430 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:47.634430 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:47.634430 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:47.634430 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:47.634430 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:47.634430 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:47.634430 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:47.634430 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:47.634430 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:47.635324 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:47.635324 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:47.635324 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:09:47.635324 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:09:47.635324 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:09:47.635324 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:09:47.635324 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:09:47.635324 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:09:47.635324 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 03:09:47.635324 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:47.637102 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 275/554] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 03:09:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 471 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:09:47.640748 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-5337d050-af59-430a-b1d3-4e0ef1c8389c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 03:09:47.640748 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:47.640748 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:47.640748 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:47.640748 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:47.640748 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:47.640748 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:47.640748 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:47.640748 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:47.640748 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:47.640748 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:47.640748 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:47.640748 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:09:47.640748 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:09:47.640748 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:09:47.640748 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:09:47.641714 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:47.641714 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:47.641714 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:09:47.641714 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:09:47.641714 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:09:47.641714 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:09:47.641714 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:47.641714 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:47.641714 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:47.641714 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:47.641714 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:47.641714 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:47.641714 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:47.641714 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:47.641714 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:47.641714 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:47.641714 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:47.641714 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:47.643038 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:47.643038 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:47.643038 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:09:47.643038 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:09:47.643038 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:09:47.643038 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:09:47.643038 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:09:47.643038 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:09:47.643038 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 03:09:47.643038 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:47.643038 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 280/555] 10.222.0.50 () {56 vars in 1093 bytes} [Fri May 27 03:09:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:09:47.643709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4f671-62fb-46b3-bc28-cab5c8374af6 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:47.643709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4f671-62fb-46b3-bc28-cab5c8374af6 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:47.643709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4f671-62fb-46b3-bc28-cab5c8374af6 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:47.645160 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 281/556] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:47.649293 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca4f671-62fb-46b3-bc28-cab5c8374af6 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:47.653982 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37403c9-9abc-4d85-98dc-31c851e6b119 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:47.736436 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:47.770827 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca4f671-62fb-46b3-bc28-cab5c8374af6 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:47.782601 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37403c9-9abc-4d85-98dc-31c851e6b119 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:47.795217 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:47.815621 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c6124-0157-4c03-ac50-bb207702f18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:47.816295 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c6124-0157-4c03-ac50-bb207702f18b None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:47.816574 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c6124-0157-4c03-ac50-bb207702f18b None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:47.816846 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c6124-0157-4c03-ac50-bb207702f18b None admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:47.817770 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c6124-0157-4c03-ac50-bb207702f18b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:47.818073 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c6124-0157-4c03-ac50-bb207702f18b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:47.818234 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2960d5d-94df-4a2e-8754-98b96e856772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:47.818911 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2960d5d-94df-4a2e-8754-98b96e856772 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:47.819193 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2960d5d-94df-4a2e-8754-98b96e856772 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:47.819468 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2960d5d-94df-4a2e-8754-98b96e856772 None admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:47.819586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c6124-0157-4c03-ac50-bb207702f18b None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:47.820311 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2960d5d-94df-4a2e-8754-98b96e856772 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:47.820616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2960d5d-94df-4a2e-8754-98b96e856772 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:47.822086 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2960d5d-94df-4a2e-8754-98b96e856772 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:47.838551 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 282/557] 10.222.0.50 () {56 vars in 1135 bytes} [Fri May 27 03:09:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:47.840348 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85def60c-7365-4489-af73-6c9a39381fcc None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:47.840635 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85def60c-7365-4489-af73-6c9a39381fcc None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:47.840907 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85def60c-7365-4489-af73-6c9a39381fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:47.857672 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-85def60c-7365-4489-af73-6c9a39381fcc None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:47.859069 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-85def60c-7365-4489-af73-6c9a39381fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:47.864677 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 276/558] 10.222.0.50 () {56 vars in 1133 bytes} [Fri May 27 03:09:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:47.865884 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458aac5-221e-4c22-ba16-ecdf7c31c67c None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:47.866179 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458aac5-221e-4c22-ba16-ecdf7c31c67c None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:47.866457 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458aac5-221e-4c22-ba16-ecdf7c31c67c None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:47.871874 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 277/559] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:47.872653 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d458aac5-221e-4c22-ba16-ecdf7c31c67c None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:47.937004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 283/560] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:47.937004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85def60c-7365-4489-af73-6c9a39381fcc None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:48.013566 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d458aac5-221e-4c22-ba16-ecdf7c31c67c None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:48.034358 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:48.043385 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85def60c-7365-4489-af73-6c9a39381fcc None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:48.046829 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:48.054249 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34551d69-af24-4512-836d-5beabbf4088a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:48.054910 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34551d69-af24-4512-836d-5beabbf4088a None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:48.055192 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34551d69-af24-4512-836d-5beabbf4088a None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:48.055465 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34551d69-af24-4512-836d-5beabbf4088a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:48.080183 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5799c597-025a-4f28-99d3-65ac9dafba33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:48.081015 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799c597-025a-4f28-99d3-65ac9dafba33 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:48.081304 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799c597-025a-4f28-99d3-65ac9dafba33 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:48.081573 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799c597-025a-4f28-99d3-65ac9dafba33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:48.092560 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5799c597-025a-4f28-99d3-65ac9dafba33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:48.094081 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5799c597-025a-4f28-99d3-65ac9dafba33 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:48.111415 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-5799c597-025a-4f28-99d3-65ac9dafba33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:48.111415 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:48.111415 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:48.111415 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:48.111415 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:48.111415 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:48.111415 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:48.111415 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:48.111415 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:48.111415 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:48.111415 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:48.111415 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:48.111415 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:09:48.111415 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:09:48.111415 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:09:48.112506 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:09:48.112506 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:48.112506 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:48.112506 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:48.112506 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:48.112506 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:48.112506 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:48.112506 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:48.112506 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:48.112506 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:48.112506 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:48.112506 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:48.112506 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:48.112506 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:48.112506 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:48.112506 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:48.112506 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:48.113642 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:09:48.113642 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:09:48.113642 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:48.113642 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:48.117389 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 284/561] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:48.118851 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1e4b7-c6ee-4418-8077-290861a66b4c None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:48.119140 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1e4b7-c6ee-4418-8077-290861a66b4c None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:48.119416 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1e4b7-c6ee-4418-8077-290861a66b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:48.128898 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34551d69-af24-4512-836d-5beabbf4088a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:48.130650 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34551d69-af24-4512-836d-5beabbf4088a None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:48.135697 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-f6a1e4b7-c6ee-4418-8077-290861a66b4c None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:48.136863 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-f6a1e4b7-c6ee-4418-8077-290861a66b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:48.203035 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 285/562] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:48.204143 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1e4b7-c6ee-4418-8077-290861a66b4c None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:48.239768 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 278/563] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:09:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:09:48.239768 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34551d69-af24-4512-836d-5beabbf4088a None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:48.255143 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:48.290935 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:48.306270 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1e4b7-c6ee-4418-8077-290861a66b4c None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:48.339448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e442bb1-3571-4654-9c86-5aeb07489a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:48.340257 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e442bb1-3571-4654-9c86-5aeb07489a3c None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:48.340534 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e442bb1-3571-4654-9c86-5aeb07489a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:48.340811 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e442bb1-3571-4654-9c86-5aeb07489a3c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:48.342733 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34551d69-af24-4512-836d-5beabbf4088a None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:48.350794 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e442bb1-3571-4654-9c86-5aeb07489a3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:48.358446 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-5e442bb1-3571-4654-9c86-5aeb07489a3c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:09:48.358446 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:48.358446 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:48.358446 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:48.358446 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:48.358446 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:48.358446 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:48.358446 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:48.358446 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:48.358446 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:48.358446 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:48.358446 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:48.358446 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:48.358446 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:48.358446 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:48.358446 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:48.359594 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:48.359594 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:48.359594 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:48.359594 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:48.359594 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:48.359594 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:48.359594 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:48.359594 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:48.359594 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:48.359594 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:48.359594 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:48.359594 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:48.359594 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:48.359594 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:48.359594 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:48.359594 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:48.360623 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:48.360623 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:48.360623 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:48.360623 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:48.360623 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:48.360623 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:48.360623 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:48.360623 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:48.360623 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:48.360623 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:48.360623 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:09:48.360623 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:48.360623 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 286/564] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 03:09:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:48.361543 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984c562-2ed8-48a6-ad0c-b464336379f8 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:48.361543 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984c562-2ed8-48a6-ad0c-b464336379f8 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:48.361543 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984c562-2ed8-48a6-ad0c-b464336379f8 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:48.362878 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 287/565] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:48.363412 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984c562-2ed8-48a6-ad0c-b464336379f8 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:48.384333 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bad5fd-3135-4245-b5ba-13dec43d4c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:48.385029 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bad5fd-3135-4245-b5ba-13dec43d4c6c None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:48.385308 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bad5fd-3135-4245-b5ba-13dec43d4c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:48.385586 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bad5fd-3135-4245-b5ba-13dec43d4c6c None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:48.386305 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bad5fd-3135-4245-b5ba-13dec43d4c6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:48.386605 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bad5fd-3135-4245-b5ba-13dec43d4c6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:48.387987 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bad5fd-3135-4245-b5ba-13dec43d4c6c None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:48.401315 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 279/566] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:09:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:48.402113 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bad5fd-3135-4245-b5ba-13dec43d4c6c None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:48.477712 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984c562-2ed8-48a6-ad0c-b464336379f8 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:48.495404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:48.514333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8c9dd-03f7-479b-957e-27ffc143eee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:48.515142 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8c9dd-03f7-479b-957e-27ffc143eee2 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:48.515421 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8c9dd-03f7-479b-957e-27ffc143eee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:48.515689 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8c9dd-03f7-479b-957e-27ffc143eee2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:48.518821 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bad5fd-3135-4245-b5ba-13dec43d4c6c None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:48.526149 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b8c9dd-03f7-479b-957e-27ffc143eee2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:48.533709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-24b8c9dd-03f7-479b-957e-27ffc143eee2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:09:48.533709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:48.533709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:48.533709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:48.533709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:48.533709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:48.533709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:48.533709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:48.533709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:48.533709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:48.533709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:48.533709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:48.533709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:48.533709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:48.533709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:48.533709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:48.535004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:48.535004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:48.535004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:48.535004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:48.535004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:48.535004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:48.535004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:48.535004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:48.535004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:48.535004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:48.535004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:48.535004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:48.535004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:48.535004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:48.535004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:48.535004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:48.535897 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:48.535897 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:48.535897 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:48.535897 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:48.535897 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:48.535897 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:48.535897 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:48.535897 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:48.535897 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:48.535897 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:48.535897 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:09:48.535897 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:48.535897 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 288/567] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 03:09:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:48.536580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8c9dd-03f7-479b-957e-27ffc143eee2 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:48.545740 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:48.558102 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2370605c-e2c1-4b83-831a-d8d577bf5eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:48.558908 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370605c-e2c1-4b83-831a-d8d577bf5eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:48.559186 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370605c-e2c1-4b83-831a-d8d577bf5eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:48.559467 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2370605c-e2c1-4b83-831a-d8d577bf5eb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:48.570359 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2370605c-e2c1-4b83-831a-d8d577bf5eb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:48.571812 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2370605c-e2c1-4b83-831a-d8d577bf5eb8 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:48.588807 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-2370605c-e2c1-4b83-831a-d8d577bf5eb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:48.588807 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:48.588807 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:48.588807 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:48.588807 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:48.588807 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:48.588807 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:48.588807 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:48.588807 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:48.588807 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:48.588807 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:48.588807 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:48.588807 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:09:48.588807 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:09:48.588807 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:09:48.589908 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:09:48.589908 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:48.589908 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:48.589908 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:48.589908 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:48.589908 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:48.589908 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:48.589908 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:48.589908 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:48.589908 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:48.589908 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:48.589908 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:48.589908 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:48.589908 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:48.589908 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:48.589908 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:48.589908 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:48.591010 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:09:48.591010 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:09:48.591010 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:48.591010 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:48.591010 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 280/568] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:48.591517 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af364c-cd15-4c77-86e9-b75f4b9998fa None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:48.591677 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af364c-cd15-4c77-86e9-b75f4b9998fa None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:48.591953 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af364c-cd15-4c77-86e9-b75f4b9998fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:48.609047 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-b3af364c-cd15-4c77-86e9-b75f4b9998fa None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:48.610239 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-b3af364c-cd15-4c77-86e9-b75f4b9998fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:48.640869 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8c9dd-03f7-479b-957e-27ffc143eee2 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:48.674670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0febf2bf-5c10-4340-b860-11d6f893a2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:48.675357 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0febf2bf-5c10-4340-b860-11d6f893a2a7 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:48.675634 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0febf2bf-5c10-4340-b860-11d6f893a2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:48.675904 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0febf2bf-5c10-4340-b860-11d6f893a2a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:48.676655 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0febf2bf-5c10-4340-b860-11d6f893a2a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:48.676977 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0febf2bf-5c10-4340-b860-11d6f893a2a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:48.678379 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0febf2bf-5c10-4340-b860-11d6f893a2a7 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:48.679720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 281/569] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:48.680974 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3af364c-cd15-4c77-86e9-b75f4b9998fa None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:48.691721 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 289/570] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 03:09:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:48.691721 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0febf2bf-5c10-4340-b860-11d6f893a2a7 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:48.741271 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:48.757802 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:48.783049 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3af364c-cd15-4c77-86e9-b75f4b9998fa None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:48.801473 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0febf2bf-5c10-4340-b860-11d6f893a2a7 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:48.818610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947dc87f-3e5b-4f1f-8376-b2f5862b62cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:48.819418 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947dc87f-3e5b-4f1f-8376-b2f5862b62cc None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:48.819700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947dc87f-3e5b-4f1f-8376-b2f5862b62cc None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:48.819977 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947dc87f-3e5b-4f1f-8376-b2f5862b62cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:48.826655 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947dc87f-3e5b-4f1f-8376-b2f5862b62cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:48.828132 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947dc87f-3e5b-4f1f-8376-b2f5862b62cc None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:48.835449 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a468d502-f3eb-4c4d-8b71-96f985ca2f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:48.836118 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468d502-f3eb-4c4d-8b71-96f985ca2f68 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:48.836396 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468d502-f3eb-4c4d-8b71-96f985ca2f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:48.836663 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468d502-f3eb-4c4d-8b71-96f985ca2f68 None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:48.837435 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a468d502-f3eb-4c4d-8b71-96f985ca2f68 None admin] RBAC: Adding query filter params (name=service) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:48.837735 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a468d502-f3eb-4c4d-8b71-96f985ca2f68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:48.839102 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a468d502-f3eb-4c4d-8b71-96f985ca2f68 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:48.841711 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-947dc87f-3e5b-4f1f-8376-b2f5862b62cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:09:48.841711 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:48.841711 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:48.841711 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:48.841711 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:48.841711 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:48.841711 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:48.841711 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:48.841711 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:48.841711 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:48.841711 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:48.841711 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:48.841711 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:09:48.841711 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:09:48.841711 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:09:48.842761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:09:48.842761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:48.842761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:48.842761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:48.842761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:48.842761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:48.842761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:48.842761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:48.842761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:48.842761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:48.842761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:48.842761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:48.842761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:48.842761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:48.842761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:48.842761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:48.842761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:48.843732 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:09:48.843732 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:09:48.843732 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:09:48.843732 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:09:48.843732 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:09:48.843732 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:09:48.843732 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:09:48.843732 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:48.843732 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 282/571] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:09:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:48.843732 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947dc87f-3e5b-4f1f-8376-b2f5862b62cc None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:48.848581 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 290/572] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 03:09:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:48.849692 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a468d502-f3eb-4c4d-8b71-96f985ca2f68 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:48.945152 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947dc87f-3e5b-4f1f-8376-b2f5862b62cc None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:48.954169 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a468d502-f3eb-4c4d-8b71-96f985ca2f68 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:48.966526 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:48.980793 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2d256-8d22-4009-94cb-c7e490d6b9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:48.981495 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2d256-8d22-4009-94cb-c7e490d6b9b3 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:48.981773 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2d256-8d22-4009-94cb-c7e490d6b9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:48.982046 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2d256-8d22-4009-94cb-c7e490d6b9b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:48.982772 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f2d256-8d22-4009-94cb-c7e490d6b9b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:48.983083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f2d256-8d22-4009-94cb-c7e490d6b9b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:48.983679 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:48.984455 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f2d256-8d22-4009-94cb-c7e490d6b9b3 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:48.989902 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3ff57-103b-47ff-951c-5fb64aaccfb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:48.990701 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3ff57-103b-47ff-951c-5fb64aaccfb9 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:48.990973 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3ff57-103b-47ff-951c-5fb64aaccfb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:48.991238 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3ff57-103b-47ff-951c-5fb64aaccfb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:48.997828 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 283/573] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:09:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:48.998633 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2d256-8d22-4009-94cb-c7e490d6b9b3 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:49.002082 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3ff57-103b-47ff-951c-5fb64aaccfb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:49.003519 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3ff57-103b-47ff-951c-5fb64aaccfb9 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:49.020461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-7ba3ff57-103b-47ff-951c-5fb64aaccfb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:49.020461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:49.020461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:49.020461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:49.020461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:49.020461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:49.020461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:49.020461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:49.020461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:49.020461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:49.020461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:49.020461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:49.020461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:09:49.020461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:09:49.020461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:09:49.021626 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:09:49.021626 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:49.021626 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:49.021626 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:49.021626 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:49.021626 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:49.021626 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:49.021626 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:49.021626 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:49.021626 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:49.021626 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:49.021626 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:49.021626 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:49.021626 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:49.021626 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:49.021626 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:49.021626 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:49.022573 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:09:49.022573 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:09:49.022573 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:49.022573 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:49.022573 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 291/574] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:49.022573 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3ff57-103b-47ff-951c-5fb64aaccfb9 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:49.100338 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2d256-8d22-4009-94cb-c7e490d6b9b3 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:49.121484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3ff57-103b-47ff-951c-5fb64aaccfb9 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:49.134348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1e216-daab-4238-97b0-8aacb84e211a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:49.135148 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1e216-daab-4238-97b0-8aacb84e211a None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:49.135426 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1e216-daab-4238-97b0-8aacb84e211a None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:49.135697 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1e216-daab-4238-97b0-8aacb84e211a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:49.145891 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af1e216-daab-4238-97b0-8aacb84e211a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:49.153788 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-5af1e216-daab-4238-97b0-8aacb84e211a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:09:49.153788 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:49.153788 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:49.153788 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:49.153788 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:49.153788 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:49.153788 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:49.153788 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:49.153788 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:49.153788 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:49.153788 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:49.153788 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:49.153788 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:49.153788 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:49.153788 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:49.153788 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:49.154725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:49.154725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:49.154725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:49.154725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:49.154725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:49.154725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:49.154725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:49.154725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:49.154725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:49.154725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:49.154725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:49.154725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:49.154725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:49.154725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:49.154725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:49.154725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:49.155574 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:49.155574 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:49.155574 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:49.155574 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:49.155574 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:49.155574 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:49.155574 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:49.155574 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:49.155574 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:49.155574 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:49.155574 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:09:49.155574 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:49.156569 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d8216-5b03-495c-92f2-3b6f276466b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:49.156569 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d8216-5b03-495c-92f2-3b6f276466b5 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:49.156731 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 284/575] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 03:09:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:49.156818 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d8216-5b03-495c-92f2-3b6f276466b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:49.156818 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d8216-5b03-495c-92f2-3b6f276466b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:49.156940 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1e216-daab-4238-97b0-8aacb84e211a None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:49.157054 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d8216-5b03-495c-92f2-3b6f276466b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:49.157054 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d8216-5b03-495c-92f2-3b6f276466b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:49.157899 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d8216-5b03-495c-92f2-3b6f276466b5 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:49.171542 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 292/576] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 03:09:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:49.172339 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d8216-5b03-495c-92f2-3b6f276466b5 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:49.201044 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:49.216144 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:49.260382 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1e216-daab-4238-97b0-8aacb84e211a None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:49.271077 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d8216-5b03-495c-92f2-3b6f276466b5 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:49.295650 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c88be41-c814-43b7-a771-b343f3129b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:49.296445 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88be41-c814-43b7-a771-b343f3129b6f None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:49.296722 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88be41-c814-43b7-a771-b343f3129b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:49.297018 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88be41-c814-43b7-a771-b343f3129b6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:49.307721 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c88be41-c814-43b7-a771-b343f3129b6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:49.308659 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2dd33f-c9d2-4e8f-99ac-46fefe3770b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:49.309232 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c88be41-c814-43b7-a771-b343f3129b6f None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:49.309503 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2dd33f-c9d2-4e8f-99ac-46fefe3770b5 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:49.309781 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2dd33f-c9d2-4e8f-99ac-46fefe3770b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:49.310054 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2dd33f-c9d2-4e8f-99ac-46fefe3770b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:49.317147 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2dd33f-c9d2-4e8f-99ac-46fefe3770b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:49.318738 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2dd33f-c9d2-4e8f-99ac-46fefe3770b5 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:49.326047 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-3c88be41-c814-43b7-a771-b343f3129b6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:49.326047 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:49.326047 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:49.326047 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:49.326047 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:49.326047 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:49.326047 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:49.326047 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:49.326047 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:49.326047 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:49.326047 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:49.326047 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:49.326047 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:09:49.326047 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:09:49.326047 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:09:49.327083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:09:49.327083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:49.327083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:49.327083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:49.327083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:49.327083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:49.327083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:49.327083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:49.327083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:49.327083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:49.327083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:49.327083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:49.327083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:49.327083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:49.327083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:49.327083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:49.327083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:49.328014 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:09:49.328014 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:09:49.328014 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:49.328014 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:49.328014 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 285/577] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:49.328014 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c88be41-c814-43b7-a771-b343f3129b6f None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:49.332445 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-9d2dd33f-c9d2-4e8f-99ac-46fefe3770b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:09:49.332445 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:49.332445 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:49.332445 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:49.332445 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:49.332445 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:49.332445 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:49.332445 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:49.332445 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:49.332445 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:49.332445 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:49.332445 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:49.332445 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:09:49.332445 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:09:49.332445 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:09:49.333356 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:09:49.333356 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:49.333356 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:49.333356 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:49.333356 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:49.333356 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:49.333356 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:49.333356 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:49.333356 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:49.333356 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:49.333356 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:49.333356 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:49.333356 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:49.333356 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:49.333356 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:49.333356 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:49.333356 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:49.334235 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:09:49.334235 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:09:49.334235 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:09:49.334235 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:09:49.334235 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:09:49.334235 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:09:49.334235 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:09:49.334235 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:49.334235 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 293/578] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:09:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:49.334992 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2dd33f-c9d2-4e8f-99ac-46fefe3770b5 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:49.423612 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:49.433836 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c88be41-c814-43b7-a771-b343f3129b6f None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:49.443976 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2dd33f-c9d2-4e8f-99ac-46fefe3770b5 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:49.445582 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:49.467474 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6936b44-2539-4c81-b1be-12a15e839d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:49.468133 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6936b44-2539-4c81-b1be-12a15e839d7d None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:49.468406 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6936b44-2539-4c81-b1be-12a15e839d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:49.468679 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6936b44-2539-4c81-b1be-12a15e839d7d None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:49.469463 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6936b44-2539-4c81-b1be-12a15e839d7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:49.469764 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6936b44-2539-4c81-b1be-12a15e839d7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:49.471175 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6936b44-2539-4c81-b1be-12a15e839d7d None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:49.480764 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267b058a-274c-4531-b5f0-1e067d413f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:49.481459 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267b058a-274c-4531-b5f0-1e067d413f23 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:49.481717 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267b058a-274c-4531-b5f0-1e067d413f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:49.481987 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267b058a-274c-4531-b5f0-1e067d413f23 None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:49.482715 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267b058a-274c-4531-b5f0-1e067d413f23 None admin] RBAC: Adding query filter params (name=service) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:49.482907 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 286/579] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:09:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:49.483006 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267b058a-274c-4531-b5f0-1e067d413f23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:49.483666 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6936b44-2539-4c81-b1be-12a15e839d7d None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:49.484372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267b058a-274c-4531-b5f0-1e067d413f23 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:49.495988 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 294/580] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 03:09:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:49.496794 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267b058a-274c-4531-b5f0-1e067d413f23 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:49.582859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6936b44-2539-4c81-b1be-12a15e839d7d None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:49.593872 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267b058a-274c-4531-b5f0-1e067d413f23 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:49.616050 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f85cfd-801e-45a0-bf8b-49af186b125c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:49.616975 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f85cfd-801e-45a0-bf8b-49af186b125c None admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:49.617254 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f85cfd-801e-45a0-bf8b-49af186b125c None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:49.617526 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f85cfd-801e-45a0-bf8b-49af186b125c None admin] PATH_INFO: `/v3/projects/f8398941252346468483ea0c863122f4/users/1a699988f08a4001b3b7d764f01f4fc5/roles/3451aa51e64642768b2228126d20fa03` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:49.627737 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eefbf06-1d3e-449e-be02-e3154900acf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:49.628396 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eefbf06-1d3e-449e-be02-e3154900acf6 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:49.628677 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eefbf06-1d3e-449e-be02-e3154900acf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:49.628975 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eefbf06-1d3e-449e-be02-e3154900acf6 None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:49.630085 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eefbf06-1d3e-449e-be02-e3154900acf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:49.630233 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eefbf06-1d3e-449e-be02-e3154900acf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:49.631588 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eefbf06-1d3e-449e-be02-e3154900acf6 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:49.632012 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f85cfd-801e-45a0-bf8b-49af186b125c None admin] RBAC: Authorizing `identity:create_grant(project_id=f8398941252346468483ea0c863122f4, user_id=1a699988f08a4001b3b7d764f01f4fc5, role_id=3451aa51e64642768b2228126d20fa03)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:49.633494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f85cfd-801e-45a0-bf8b-49af186b125c None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:49.643985 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 295/581] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:09:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:49.644649 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eefbf06-1d3e-449e-be02-e3154900acf6 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:49.651347 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 287/582] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 03:09:49 2022] PUT /identity/v3/projects/f8398941252346468483ea0c863122f4/users/1a699988f08a4001b3b7d764f01f4fc5/roles/3451aa51e64642768b2228126d20fa03 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:09:49.652268 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f85cfd-801e-45a0-bf8b-49af186b125c None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:49.654995 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:49.676675 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:49.759529 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eefbf06-1d3e-449e-be02-e3154900acf6 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:49.783170 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f85cfd-801e-45a0-bf8b-49af186b125c None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:49.797182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4add707-0cbd-419d-84d1-6dd0cc96f96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:49.797871 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4add707-0cbd-419d-84d1-6dd0cc96f96c None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:49.798175 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4add707-0cbd-419d-84d1-6dd0cc96f96c None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:49.798428 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4add707-0cbd-419d-84d1-6dd0cc96f96c None admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:49.799225 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4add707-0cbd-419d-84d1-6dd0cc96f96c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:49.799537 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4add707-0cbd-419d-84d1-6dd0cc96f96c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:49.801059 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4add707-0cbd-419d-84d1-6dd0cc96f96c None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:49.814054 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 296/583] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 03:09:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:49.814923 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4add707-0cbd-419d-84d1-6dd0cc96f96c None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:49.827655 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734990fb-01ea-4d27-b74e-687c10db452a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:49.828820 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734990fb-01ea-4d27-b74e-687c10db452a None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:49.829094 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734990fb-01ea-4d27-b74e-687c10db452a None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:49.829373 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734990fb-01ea-4d27-b74e-687c10db452a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:49.837882 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734990fb-01ea-4d27-b74e-687c10db452a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:49.839605 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734990fb-01ea-4d27-b74e-687c10db452a None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:49.856855 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-734990fb-01ea-4d27-b74e-687c10db452a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 27 03:09:49.856855 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:49.856855 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:49.856855 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:49.856855 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:49.856855 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:49.856855 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:49.856855 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:49.856855 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:49.856855 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:49.856855 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:49.856855 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:49.856855 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:09:49.856855 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:09:49.856855 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:09:49.856855 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:09:49.858030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:49.858030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:49.858030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:09:49.858030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:09:49.858030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:09:49.858030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:09:49.858030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:49.858030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:49.858030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:49.858030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:49.858030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:49.858030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:49.858030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:49.858030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:49.858030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:49.858030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:49.858030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:49.858030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:49.858980 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:49.858980 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:49.858980 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:09:49.858980 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:09:49.858980 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:09:49.858980 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:09:49.858980 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:09:49.858980 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:09:49.858980 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 27 03:09:49.858980 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:49.858980 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 288/584] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:09:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:49.858980 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734990fb-01ea-4d27-b74e-687c10db452a None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:49.916985 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:49.924504 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:49.934187 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4add707-0cbd-419d-84d1-6dd0cc96f96c None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:49.965902 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734990fb-01ea-4d27-b74e-687c10db452a None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:49.969665 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9117af-8aac-4b00-9efd-00f7b562c220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:49.970505 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9117af-8aac-4b00-9efd-00f7b562c220 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:49.970784 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9117af-8aac-4b00-9efd-00f7b562c220 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:49.971056 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9117af-8aac-4b00-9efd-00f7b562c220 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:49.983642 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9117af-8aac-4b00-9efd-00f7b562c220 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:49.985443 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9117af-8aac-4b00-9efd-00f7b562c220 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:50.004479 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7bbc1-3e96-40b6-a688-4a4019452369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:50.005397 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7bbc1-3e96-40b6-a688-4a4019452369 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:50.005679 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7bbc1-3e96-40b6-a688-4a4019452369 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:50.005954 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7bbc1-3e96-40b6-a688-4a4019452369 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:50.008385 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-6b9117af-8aac-4b00-9efd-00f7b562c220 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:50.008385 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:50.008385 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:50.008385 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:50.008385 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:50.008385 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:50.008385 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:50.008385 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:50.008385 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:50.008385 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:50.008385 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:50.008385 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:50.008385 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:09:50.008385 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:09:50.008385 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:09:50.009255 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:09:50.009255 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:50.009255 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:50.009255 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:50.009255 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:50.009255 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:50.009255 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:50.009255 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:50.009255 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:50.009255 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:50.009255 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:50.009255 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:50.009255 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:50.009255 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:50.009255 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:50.009255 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:50.009255 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:50.010156 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:09:50.010156 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:09:50.010156 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:50.010156 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:50.010156 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 297/585] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:50.010825 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9117af-8aac-4b00-9efd-00f7b562c220 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:50.018829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7bbc1-3e96-40b6-a688-4a4019452369 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:50.020350 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7bbc1-3e96-40b6-a688-4a4019452369 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:50.035480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-0fa7bbc1-3e96-40b6-a688-4a4019452369 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 27 03:09:50.035480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:50.035480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:50.035480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:50.035480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:50.035480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:50.035480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:50.035480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:50.035480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:50.035480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:50.035480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:50.035480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:50.035480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:09:50.035480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:09:50.035480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:09:50.035480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:09:50.036844 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:50.036844 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:50.036844 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:09:50.036844 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:09:50.036844 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:09:50.036844 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:09:50.036844 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:50.036844 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:50.036844 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:50.036844 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:50.036844 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:50.036844 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:50.036844 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:50.036844 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:50.036844 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:50.036844 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:50.036844 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:50.036844 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:50.038091 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:50.038091 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:50.038091 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:09:50.038091 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:09:50.038091 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:09:50.038091 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:09:50.038091 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:09:50.038091 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:09:50.038091 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 27 03:09:50.038091 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:50.038091 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 289/586] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:09:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:50.038091 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7bbc1-3e96-40b6-a688-4a4019452369 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:50.123415 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9117af-8aac-4b00-9efd-00f7b562c220 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:50.143416 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7bbc1-3e96-40b6-a688-4a4019452369 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:50.159263 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a660eb-b6c6-4f12-9ada-d5861c642c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:50.160367 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a660eb-b6c6-4f12-9ada-d5861c642c55 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:50.160740 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a660eb-b6c6-4f12-9ada-d5861c642c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:50.161223 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a660eb-b6c6-4f12-9ada-d5861c642c55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:50.162290 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a660eb-b6c6-4f12-9ada-d5861c642c55 None admin] RBAC: Adding query filter params (role.id=3451aa51e64642768b2228126d20fa03, scope.project.id=f8398941252346468483ea0c863122f4, user.id=3de53e2f0fba4e0fac70887067c1963d) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:50.162604 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a660eb-b6c6-4f12-9ada-d5861c642c55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:50.164114 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a660eb-b6c6-4f12-9ada-d5861c642c55 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:50.167135 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:50.169256 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:50.173023 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 298/587] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:09:50 2022] GET /identity/v3/role_assignments?user.id=3de53e2f0fba4e0fac70887067c1963d&scope.project.id=f8398941252346468483ea0c863122f4&role.id=3451aa51e64642768b2228126d20fa03 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:50.173993 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a660eb-b6c6-4f12-9ada-d5861c642c55 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:50.180441 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8402fc6f-8f25-4c09-9e47-3667e9635198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:50.181327 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8402fc6f-8f25-4c09-9e47-3667e9635198 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:50.181705 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8402fc6f-8f25-4c09-9e47-3667e9635198 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:50.182076 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8402fc6f-8f25-4c09-9e47-3667e9635198 None admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:50.183055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8402fc6f-8f25-4c09-9e47-3667e9635198 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:50.183459 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8402fc6f-8f25-4c09-9e47-3667e9635198 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:50.185282 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8402fc6f-8f25-4c09-9e47-3667e9635198 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:50.207781 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 290/588] 10.222.0.50 () {56 vars in 1137 bytes} [Fri May 27 03:09:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:50.209072 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8402fc6f-8f25-4c09-9e47-3667e9635198 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:50.314425 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a660eb-b6c6-4f12-9ada-d5861c642c55 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:50.345724 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8402fc6f-8f25-4c09-9e47-3667e9635198 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:50.353177 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53e247-76aa-42b6-8d87-c70e790528fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:50.353952 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53e247-76aa-42b6-8d87-c70e790528fb None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:50.354296 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53e247-76aa-42b6-8d87-c70e790528fb None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:50.354629 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53e247-76aa-42b6-8d87-c70e790528fb None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:50.355421 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a53e247-76aa-42b6-8d87-c70e790528fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:50.356243 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a53e247-76aa-42b6-8d87-c70e790528fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:50.357435 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a53e247-76aa-42b6-8d87-c70e790528fb None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:50.375877 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 299/589] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:50.377955 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53e247-76aa-42b6-8d87-c70e790528fb None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:50.386755 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b9fb8-93ea-401f-88b5-33ccedba9aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:50.387514 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b9fb8-93ea-401f-88b5-33ccedba9aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:50.388060 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b9fb8-93ea-401f-88b5-33ccedba9aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:50.388172 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b9fb8-93ea-401f-88b5-33ccedba9aa1 None admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:50.389126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9b9fb8-93ea-401f-88b5-33ccedba9aa1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:50.389560 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9b9fb8-93ea-401f-88b5-33ccedba9aa1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:50.391117 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9b9fb8-93ea-401f-88b5-33ccedba9aa1 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:50.409850 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 291/590] 10.222.0.50 () {56 vars in 1137 bytes} [Fri May 27 03:09:50 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:50.410964 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b9fb8-93ea-401f-88b5-33ccedba9aa1 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:50.451201 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:50.454471 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:50.512078 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a53e247-76aa-42b6-8d87-c70e790528fb None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:50.521356 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b9fb8-93ea-401f-88b5-33ccedba9aa1 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:50.559787 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe35040-3b65-4646-b03a-4920cc0e5e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:50.560999 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe35040-3b65-4646-b03a-4920cc0e5e3e None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:50.561299 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f9edd-3774-461f-89e0-e53ed9f11c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:50.561658 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe35040-3b65-4646-b03a-4920cc0e5e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:50.562090 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe35040-3b65-4646-b03a-4920cc0e5e3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:50.562381 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f9edd-3774-461f-89e0-e53ed9f11c91 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:50.562754 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f9edd-3774-461f-89e0-e53ed9f11c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:50.563210 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f9edd-3774-461f-89e0-e53ed9f11c91 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:50.571417 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458f9edd-3774-461f-89e0-e53ed9f11c91 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:50.573284 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458f9edd-3774-461f-89e0-e53ed9f11c91 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:50.575746 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe35040-3b65-4646-b03a-4920cc0e5e3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:50.577943 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe35040-3b65-4646-b03a-4920cc0e5e3e None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:50.589584 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-458f9edd-3774-461f-89e0-e53ed9f11c91 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 27 03:09:50.589584 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:50.589584 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:50.589584 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:50.589584 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:50.589584 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:50.589584 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:50.589584 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:50.589584 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:50.589584 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:50.589584 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:50.589584 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:50.589584 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:09:50.589584 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:09:50.589584 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:09:50.589584 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:09:50.590763 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:50.590763 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:50.590763 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:09:50.590763 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:09:50.590763 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:09:50.590763 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:09:50.590763 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:50.590763 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:50.590763 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:50.590763 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:50.590763 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:50.590763 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:50.590763 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:50.590763 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:50.590763 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:50.590763 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:50.590763 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:50.590763 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:50.591899 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:50.591899 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:50.591899 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:09:50.591899 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:09:50.591899 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:09:50.591899 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:09:50.591899 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:09:50.591899 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:09:50.591899 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 27 03:09:50.591899 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:50.593863 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 292/591] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 03:09:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:50.594736 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f9edd-3774-461f-89e0-e53ed9f11c91 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:50.603237 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-bbe35040-3b65-4646-b03a-4920cc0e5e3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:50.603237 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:50.603237 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:50.603237 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:50.603237 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:50.603237 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:50.603237 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:50.603237 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:50.603237 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:50.603237 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:50.603237 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:50.603237 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:50.603237 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:09:50.603237 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:09:50.603237 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:09:50.604268 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:09:50.604268 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:50.604268 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:50.604268 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:50.604268 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:50.604268 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:50.604268 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:50.604268 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:50.604268 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:50.604268 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:50.604268 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:50.604268 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:50.604268 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:50.604268 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:50.604268 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:50.604268 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:50.604268 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:50.605366 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:09:50.605366 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:09:50.605366 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:50.605366 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:50.607792 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 300/592] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:50.609371 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe35040-3b65-4646-b03a-4920cc0e5e3e None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:50.703724 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:50.713088 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f9edd-3774-461f-89e0-e53ed9f11c91 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:50.745978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe35040-3b65-4646-b03a-4920cc0e5e3e None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:50.754452 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ff451-f88b-408c-9e2c-5bea07c14dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:50.755633 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ff451-f88b-408c-9e2c-5bea07c14dfc None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:50.756158 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ff451-f88b-408c-9e2c-5bea07c14dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:50.756375 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ff451-f88b-408c-9e2c-5bea07c14dfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:50.763410 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:50.768582 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737ff451-f88b-408c-9e2c-5bea07c14dfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:50.770506 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737ff451-f88b-408c-9e2c-5bea07c14dfc None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:50.790000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-737ff451-f88b-408c-9e2c-5bea07c14dfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:50.790000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:50.790000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:50.790000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:50.790000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:50.790000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:50.790000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:50.790000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:50.790000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:50.790000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:50.790000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:50.790000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:50.790000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:09:50.790000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:09:50.790000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:09:50.791115 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:09:50.791115 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:50.791115 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:50.791115 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:50.791115 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:50.791115 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:50.791115 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:50.791115 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:50.791115 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:50.791115 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:50.791115 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:50.791115 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:50.791115 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:50.791115 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:50.791115 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:50.791115 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:50.791115 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:50.792204 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:09:50.792204 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:09:50.792204 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:50.792204 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:50.792557 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e54d073-455e-4d60-b5ac-388d33b47336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:50.793425 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54d073-455e-4d60-b5ac-388d33b47336 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:50.798167 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 293/593] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:50.798167 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ff451-f88b-408c-9e2c-5bea07c14dfc None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:50.798698 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54d073-455e-4d60-b5ac-388d33b47336 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:50.799135 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54d073-455e-4d60-b5ac-388d33b47336 None admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:50.801214 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e54d073-455e-4d60-b5ac-388d33b47336 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:50.801905 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e54d073-455e-4d60-b5ac-388d33b47336 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:50.803546 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e54d073-455e-4d60-b5ac-388d33b47336 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:50.828558 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 301/594] 10.222.0.50 () {56 vars in 1139 bytes} [Fri May 27 03:09:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:50.830308 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e54d073-455e-4d60-b5ac-388d33b47336 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:50.919732 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ff451-f88b-408c-9e2c-5bea07c14dfc None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:50.950302 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e54d073-455e-4d60-b5ac-388d33b47336 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:50.957910 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdfe421-6953-47b8-93f2-b8a0ae914275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:50.959012 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdfe421-6953-47b8-93f2-b8a0ae914275 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:50.959439 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdfe421-6953-47b8-93f2-b8a0ae914275 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:50.959882 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdfe421-6953-47b8-93f2-b8a0ae914275 None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:50.960758 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdfe421-6953-47b8-93f2-b8a0ae914275 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:50.961306 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdfe421-6953-47b8-93f2-b8a0ae914275 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:50.962935 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdfe421-6953-47b8-93f2-b8a0ae914275 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:50.976554 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 294/595] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:50.977879 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdfe421-6953-47b8-93f2-b8a0ae914275 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:50.988235 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf6e351-6d6a-4a26-b023-766392945184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:50.989289 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf6e351-6d6a-4a26-b023-766392945184 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:50.990088 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf6e351-6d6a-4a26-b023-766392945184 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:50.990365 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf6e351-6d6a-4a26-b023-766392945184 None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:50.991192 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf6e351-6d6a-4a26-b023-766392945184 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:50.991542 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf6e351-6d6a-4a26-b023-766392945184 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:50.992889 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:50.993729 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf6e351-6d6a-4a26-b023-766392945184 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:51.012718 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 302/596] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:51.013640 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf6e351-6d6a-4a26-b023-766392945184 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:51.047979 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:51.091217 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdfe421-6953-47b8-93f2-b8a0ae914275 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:51.124849 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf6e351-6d6a-4a26-b023-766392945184 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:51.127482 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7457e067-8d8a-406a-9dd0-b8d98dfcf1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:51.128474 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457e067-8d8a-406a-9dd0-b8d98dfcf1d3 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:51.128911 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457e067-8d8a-406a-9dd0-b8d98dfcf1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:51.129393 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457e067-8d8a-406a-9dd0-b8d98dfcf1d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:51.140802 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7457e067-8d8a-406a-9dd0-b8d98dfcf1d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:51.142643 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7457e067-8d8a-406a-9dd0-b8d98dfcf1d3 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:51.160502 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc75cc-a968-4ea6-968d-db1cb80d218b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:51.161400 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-7457e067-8d8a-406a-9dd0-b8d98dfcf1d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:51.161400 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:51.161400 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:51.161400 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:51.161400 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:51.161400 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:51.161400 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:51.161400 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:51.161400 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:51.161400 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:51.161400 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:51.161400 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:51.161400 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:09:51.161400 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:09:51.161400 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:09:51.162731 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:09:51.162731 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:51.162731 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:51.162731 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:51.162731 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:51.162731 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:51.162731 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:51.162731 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:51.162731 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:51.162731 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:51.162731 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:51.162731 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:51.162731 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:51.162731 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:51.162731 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:51.162731 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:51.162731 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:51.163783 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:09:51.163783 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:09:51.163783 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:51.163783 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:51.164035 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc75cc-a968-4ea6-968d-db1cb80d218b None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:51.164035 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc75cc-a968-4ea6-968d-db1cb80d218b None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:51.164035 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc75cc-a968-4ea6-968d-db1cb80d218b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:51.166225 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 295/597] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:51.167344 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7457e067-8d8a-406a-9dd0-b8d98dfcf1d3 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:51.170137 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bc75cc-a968-4ea6-968d-db1cb80d218b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:51.172016 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bc75cc-a968-4ea6-968d-db1cb80d218b None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:51.188670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-65bc75cc-a968-4ea6-968d-db1cb80d218b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:09:51.188670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:51.188670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:51.188670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:51.188670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:51.188670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:51.188670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:51.188670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:51.188670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:51.188670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:51.188670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:51.188670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:51.188670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:09:51.188670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:09:51.188670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:09:51.189805 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:09:51.189805 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:51.189805 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:51.189805 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:51.189805 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:51.189805 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:51.189805 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:51.189805 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:51.189805 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:51.189805 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:51.189805 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:51.189805 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:51.189805 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:51.189805 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:51.189805 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:51.189805 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:51.189805 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:51.190972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:09:51.190972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:09:51.190972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:09:51.190972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:09:51.190972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:09:51.190972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:09:51.190972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:09:51.190972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:51.190972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 303/598] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:09:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:51.190972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc75cc-a968-4ea6-968d-db1cb80d218b None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:51.234755 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:51.277662 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7457e067-8d8a-406a-9dd0-b8d98dfcf1d3 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:51.289905 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:51.299643 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc75cc-a968-4ea6-968d-db1cb80d218b None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:51.315615 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368beec8-b794-4442-8590-cf89f7eb305e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:51.316663 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368beec8-b794-4442-8590-cf89f7eb305e None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:51.317289 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368beec8-b794-4442-8590-cf89f7eb305e None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:51.317724 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368beec8-b794-4442-8590-cf89f7eb305e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:51.324833 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368beec8-b794-4442-8590-cf89f7eb305e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:51.326492 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368beec8-b794-4442-8590-cf89f7eb305e None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:51.336381 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01710c1-6aa9-4f08-977d-279a830d62aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:51.337081 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01710c1-6aa9-4f08-977d-279a830d62aa None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:51.337401 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01710c1-6aa9-4f08-977d-279a830d62aa None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:51.337668 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01710c1-6aa9-4f08-977d-279a830d62aa None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:51.338389 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01710c1-6aa9-4f08-977d-279a830d62aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:51.338687 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01710c1-6aa9-4f08-977d-279a830d62aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:51.340070 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01710c1-6aa9-4f08-977d-279a830d62aa None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:51.342565 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-368beec8-b794-4442-8590-cf89f7eb305e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:09:51.342565 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:51.342565 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:51.342565 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:51.342565 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:51.342565 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:51.342565 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:51.342565 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:51.342565 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:51.342565 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:51.342565 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:51.342565 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:51.342565 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:09:51.342565 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:09:51.342565 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:09:51.343624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:09:51.343624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:51.343624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:51.343624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:51.343624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:51.343624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:51.343624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:51.343624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:51.343624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:51.343624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:51.343624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:51.343624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:51.343624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:51.343624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:51.343624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:51.343624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:51.343624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:51.344672 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:09:51.344672 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:09:51.344672 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:09:51.344672 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:09:51.344672 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:09:51.344672 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:09:51.344672 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:09:51.344672 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:51.346416 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 296/599] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:09:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:51.347390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368beec8-b794-4442-8590-cf89f7eb305e None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:51.357596 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 304/600] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:09:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:51.358541 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01710c1-6aa9-4f08-977d-279a830d62aa None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:51.452702 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368beec8-b794-4442-8590-cf89f7eb305e None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:51.468437 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01710c1-6aa9-4f08-977d-279a830d62aa None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:51.477554 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:51.490183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb45c8-44bd-4ba8-a7d9-0cd347585455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:51.491109 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb45c8-44bd-4ba8-a7d9-0cd347585455 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:51.491546 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb45c8-44bd-4ba8-a7d9-0cd347585455 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:51.492024 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb45c8-44bd-4ba8-a7d9-0cd347585455 None admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:51.492978 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bb45c8-44bd-4ba8-a7d9-0cd347585455 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:51.493445 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bb45c8-44bd-4ba8-a7d9-0cd347585455 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:51.495043 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bb45c8-44bd-4ba8-a7d9-0cd347585455 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:51.505860 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45de10-5b7c-4668-89dc-47b17736f8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:51.506519 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45de10-5b7c-4668-89dc-47b17736f8f7 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:51.506796 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45de10-5b7c-4668-89dc-47b17736f8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:51.507147 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45de10-5b7c-4668-89dc-47b17736f8f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:51.507930 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec45de10-5b7c-4668-89dc-47b17736f8f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:51.508235 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec45de10-5b7c-4668-89dc-47b17736f8f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:51.509649 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 297/601] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 03:09:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:51.510034 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec45de10-5b7c-4668-89dc-47b17736f8f7 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:51.511127 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb45c8-44bd-4ba8-a7d9-0cd347585455 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:51.530897 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 305/602] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 03:09:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:51.531726 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45de10-5b7c-4668-89dc-47b17736f8f7 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:51.549362 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:51.614938 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb45c8-44bd-4ba8-a7d9-0cd347585455 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:51.643783 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45de10-5b7c-4668-89dc-47b17736f8f7 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:51.649615 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d48e9a8-0ed0-4d0a-af49-594d46287f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:51.650598 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48e9a8-0ed0-4d0a-af49-594d46287f09 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:51.651023 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48e9a8-0ed0-4d0a-af49-594d46287f09 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:51.651439 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48e9a8-0ed0-4d0a-af49-594d46287f09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:51.658466 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d48e9a8-0ed0-4d0a-af49-594d46287f09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:51.660479 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d48e9a8-0ed0-4d0a-af49-594d46287f09 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:51.675601 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-4d48e9a8-0ed0-4d0a-af49-594d46287f09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:09:51.675601 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:51.675601 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:51.675601 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:51.675601 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:51.675601 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:51.675601 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:51.675601 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:51.675601 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:51.675601 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:51.675601 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:51.675601 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:51.675601 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:09:51.675601 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:09:51.675601 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:09:51.676582 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:09:51.676582 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:51.676582 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:51.676582 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:51.676582 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:51.676582 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:51.676582 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:51.676582 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:51.676582 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:51.676582 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:51.676582 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:51.676582 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:51.676582 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:51.676582 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:51.676582 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:51.676582 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:51.676582 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:51.677470 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:09:51.677470 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:09:51.677470 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:09:51.677470 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:09:51.677470 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:09:51.677470 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:09:51.677470 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:09:51.677470 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:51.679521 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a65b36-ed69-4ff2-bddc-f472a830ec49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:51.680005 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 298/603] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:09:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:51.680378 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a65b36-ed69-4ff2-bddc-f472a830ec49 None admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:51.680712 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a65b36-ed69-4ff2-bddc-f472a830ec49 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:51.680997 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a65b36-ed69-4ff2-bddc-f472a830ec49 None admin] PATH_INFO: `/v3/projects/f8398941252346468483ea0c863122f4/users/5b3c9c5c8d2e4376b1b70d974d4cb851/roles/3451aa51e64642768b2228126d20fa03` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:51.681629 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d48e9a8-0ed0-4d0a-af49-594d46287f09 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:51.700236 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a65b36-ed69-4ff2-bddc-f472a830ec49 None admin] RBAC: Authorizing `identity:create_grant(project_id=f8398941252346468483ea0c863122f4, user_id=5b3c9c5c8d2e4376b1b70d974d4cb851, role_id=3451aa51e64642768b2228126d20fa03)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:51.700236 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a65b36-ed69-4ff2-bddc-f472a830ec49 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:51.720996 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 306/604] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 03:09:51 2022] PUT /identity/v3/projects/f8398941252346468483ea0c863122f4/users/5b3c9c5c8d2e4376b1b70d974d4cb851/roles/3451aa51e64642768b2228126d20fa03 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:09:51.720996 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a65b36-ed69-4ff2-bddc-f472a830ec49 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:51.770606 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:51.831146 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:51.922937 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d48e9a8-0ed0-4d0a-af49-594d46287f09 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:51.938901 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a65b36-ed69-4ff2-bddc-f472a830ec49 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:51.991013 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003255b0-eda8-435d-8066-bf64863c149b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:51.991686 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003255b0-eda8-435d-8066-bf64863c149b None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:51.991961 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003255b0-eda8-435d-8066-bf64863c149b None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:51.992230 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003255b0-eda8-435d-8066-bf64863c149b None admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:51.993036 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003255b0-eda8-435d-8066-bf64863c149b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:51.993340 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003255b0-eda8-435d-8066-bf64863c149b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:51.994788 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003255b0-eda8-435d-8066-bf64863c149b None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:52.004302 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f575a3-1d1b-48f1-9345-bec01f3ae374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:52.005247 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f575a3-1d1b-48f1-9345-bec01f3ae374 None admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:52.005510 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f575a3-1d1b-48f1-9345-bec01f3ae374 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:52.005777 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f575a3-1d1b-48f1-9345-bec01f3ae374 None admin] PATH_INFO: `/v3/projects/f8398941252346468483ea0c863122f4/users/36dd9b94ceb74eb4a7e9b5bbd5e8d8ba/roles/3451aa51e64642768b2228126d20fa03` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:52.014762 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 307/605] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 03:09:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:52.016504 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9615d457-e5df-4d32-b3c0-d0d9bd2e141a None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:52.017040 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9615d457-e5df-4d32-b3c0-d0d9bd2e141a None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:52.017561 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9615d457-e5df-4d32-b3c0-d0d9bd2e141a None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:52.019277 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 308/606] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:52.021131 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9615d457-e5df-4d32-b3c0-d0d9bd2e141a None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:52.029073 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f575a3-1d1b-48f1-9345-bec01f3ae374 None admin] RBAC: Authorizing `identity:create_grant(project_id=f8398941252346468483ea0c863122f4, user_id=36dd9b94ceb74eb4a7e9b5bbd5e8d8ba, role_id=3451aa51e64642768b2228126d20fa03)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:52.031471 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f575a3-1d1b-48f1-9345-bec01f3ae374 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:52.052340 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 299/607] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 03:09:51 2022] PUT /identity/v3/projects/f8398941252346468483ea0c863122f4/users/36dd9b94ceb74eb4a7e9b5bbd5e8d8ba/roles/3451aa51e64642768b2228126d20fa03 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:09:52.054385 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0d983-6739-4cab-9ef7-187dd87dd207 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:52.054928 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0d983-6739-4cab-9ef7-187dd87dd207 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:52.055459 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0d983-6739-4cab-9ef7-187dd87dd207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:52.088698 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-a3c0d983-6739-4cab-9ef7-187dd87dd207 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:52.090449 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-a3c0d983-6739-4cab-9ef7-187dd87dd207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:52.172151 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 300/608] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:52.198456 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c0d983-6739-4cab-9ef7-187dd87dd207 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:52.228388 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:52.241759 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9615d457-e5df-4d32-b3c0-d0d9bd2e141a None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:52.269403 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:52.295176 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9bd3e3-fcfe-4773-a362-c186350bf322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:52.295996 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9bd3e3-fcfe-4773-a362-c186350bf322 None admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:52.296294 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9bd3e3-fcfe-4773-a362-c186350bf322 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:52.296979 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9bd3e3-fcfe-4773-a362-c186350bf322 None admin] PATH_INFO: `/v3/projects/f8398941252346468483ea0c863122f4/users/4ee30c0d6d694ee8945800157a9fc5b7/roles/3451aa51e64642768b2228126d20fa03` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:52.315901 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9bd3e3-fcfe-4773-a362-c186350bf322 None admin] RBAC: Authorizing `identity:create_grant(project_id=f8398941252346468483ea0c863122f4, user_id=4ee30c0d6d694ee8945800157a9fc5b7, role_id=3451aa51e64642768b2228126d20fa03)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:52.317779 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9bd3e3-fcfe-4773-a362-c186350bf322 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:52.341102 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 309/609] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 03:09:52 2022] PUT /identity/v3/projects/f8398941252346468483ea0c863122f4/users/4ee30c0d6d694ee8945800157a9fc5b7/roles/3451aa51e64642768b2228126d20fa03 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:09:52.359820 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c0d983-6739-4cab-9ef7-187dd87dd207 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:52.400247 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06d6c8-9113-4174-9f21-5e78f5f3bcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:52.401218 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06d6c8-9113-4174-9f21-5e78f5f3bcfe None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:52.401567 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06d6c8-9113-4174-9f21-5e78f5f3bcfe None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:52.401910 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06d6c8-9113-4174-9f21-5e78f5f3bcfe None admin] PATH_INFO: `/v3/roles/service` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:52.419470 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e06d6c8-9113-4174-9f21-5e78f5f3bcfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:52.428095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-6e06d6c8-9113-4174-9f21-5e78f5f3bcfe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:09:52.428095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:52.428095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:52.428095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:52.428095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:52.428095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:52.428095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:52.428095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:52.428095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:52.428095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:52.428095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:52.428095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:52.428095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:52.428095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:52.428095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:52.428095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:52.429343 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:52.429343 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:52.429343 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:52.429343 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:52.429343 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:52.429343 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:52.429343 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:52.429343 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:52.429343 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:52.429343 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:52.429343 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:52.429343 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:52.429343 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:52.429343 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:52.429343 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:52.429343 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:52.430378 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:52.430378 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:52.430378 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:52.430378 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:52.430378 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:52.430378 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:52.430378 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:52.430378 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:52.430378 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:52.430378 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:52.430378 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:09:52.430378 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:52.431352 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 301/610] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 03:09:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:52.437728 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9bd3e3-fcfe-4773-a362-c186350bf322 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:52.591418 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9bd3e3-fcfe-4773-a362-c186350bf322 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:52.627857 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91801372-e959-4a46-82db-93eb7b341a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:52.628536 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91801372-e959-4a46-82db-93eb7b341a3c None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:52.628819 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91801372-e959-4a46-82db-93eb7b341a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:52.629252 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91801372-e959-4a46-82db-93eb7b341a3c None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:52.630413 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91801372-e959-4a46-82db-93eb7b341a3c None admin] RBAC: Adding query filter params (name=service) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:52.630413 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91801372-e959-4a46-82db-93eb7b341a3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:52.631806 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91801372-e959-4a46-82db-93eb7b341a3c None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:52.642714 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 310/611] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 03:09:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:52.649031 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06d6c8-9113-4174-9f21-5e78f5f3bcfe None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:52.743605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ce952-8691-4236-b3ed-311dd66ba603 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:52.743894 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ce952-8691-4236-b3ed-311dd66ba603 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:52.744182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36ce952-8691-4236-b3ed-311dd66ba603 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:52.745736 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 311/612] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:52.751920 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01467a8-6dd3-44ef-873f-7fca9a88b981 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:52.752208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01467a8-6dd3-44ef-873f-7fca9a88b981 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:52.752487 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01467a8-6dd3-44ef-873f-7fca9a88b981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:52.782697 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:52.785443 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-a01467a8-6dd3-44ef-873f-7fca9a88b981 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:52.786928 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-a01467a8-6dd3-44ef-873f-7fca9a88b981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:52.813580 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06d6c8-9113-4174-9f21-5e78f5f3bcfe None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:52.849790 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:52.870521 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059ac71-b2c2-4547-a761-9070bb8d1769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:52.871342 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059ac71-b2c2-4547-a761-9070bb8d1769 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:52.871622 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059ac71-b2c2-4547-a761-9070bb8d1769 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:52.871897 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059ac71-b2c2-4547-a761-9070bb8d1769 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:52.884604 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3059ac71-b2c2-4547-a761-9070bb8d1769 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:52.887024 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3059ac71-b2c2-4547-a761-9070bb8d1769 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:52.906856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-3059ac71-b2c2-4547-a761-9070bb8d1769 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:52.906856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:52.906856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:52.906856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:52.906856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:52.906856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:52.906856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:52.906856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:52.906856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:52.906856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:52.906856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:52.906856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:52.906856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:09:52.906856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:09:52.906856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:09:52.907935 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:09:52.907935 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:52.907935 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:52.907935 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:52.907935 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:52.907935 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:52.907935 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:52.907935 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:52.907935 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:52.907935 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:52.907935 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:52.907935 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:52.907935 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:52.907935 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:52.907935 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:52.907935 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:52.907935 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:52.909015 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:09:52.909015 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:09:52.909015 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:52.909015 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:52.911805 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 302/613] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:52.917419 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059ac71-b2c2-4547-a761-9070bb8d1769 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:52.935585 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 312/614] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 03:09:52.976680 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01467a8-6dd3-44ef-873f-7fca9a88b981 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:53.046050 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059ac71-b2c2-4547-a761-9070bb8d1769 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:53.108072 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d47129b-8af7-4fcf-b354-9d0ab4ade1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:53.109071 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47129b-8af7-4fcf-b354-9d0ab4ade1a9 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:53.109546 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47129b-8af7-4fcf-b354-9d0ab4ade1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:53.110961 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47129b-8af7-4fcf-b354-9d0ab4ade1a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:53.111873 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d47129b-8af7-4fcf-b354-9d0ab4ade1a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:53.112496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d47129b-8af7-4fcf-b354-9d0ab4ade1a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:53.115558 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d47129b-8af7-4fcf-b354-9d0ab4ade1a9 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:53.117691 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:53.139978 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 303/615] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:09:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:53.146470 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d47129b-8af7-4fcf-b354-9d0ab4ade1a9 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:53.172041 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01467a8-6dd3-44ef-873f-7fca9a88b981 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:53.233031 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6fd068-ff7a-45d1-9894-6a5c37992abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:53.234021 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fd068-ff7a-45d1-9894-6a5c37992abc None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:53.234388 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fd068-ff7a-45d1-9894-6a5c37992abc None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:53.234745 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fd068-ff7a-45d1-9894-6a5c37992abc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:53.253157 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6fd068-ff7a-45d1-9894-6a5c37992abc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:53.261933 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-af6fd068-ff7a-45d1-9894-6a5c37992abc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:09:53.261933 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:53.261933 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:53.261933 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:53.261933 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:53.261933 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:53.261933 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:53.261933 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:53.261933 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:53.261933 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:53.261933 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:53.261933 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:53.261933 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:53.261933 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:53.261933 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:53.261933 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:53.263058 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:53.263058 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:53.263058 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:53.263058 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:53.263058 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:53.263058 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:53.263058 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:53.263058 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:53.263058 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:53.263058 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:53.263058 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:53.263058 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:53.263058 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:53.263058 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:53.263058 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:53.263058 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:53.264079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:53.264079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:53.264079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:53.264079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:53.264079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:53.264079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:53.264079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:53.264079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:53.264079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:53.264079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:53.264079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:09:53.264079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:53.274521 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 313/616] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 03:09:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:09:53.278969 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6fd068-ff7a-45d1-9894-6a5c37992abc None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:53.323204 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d47129b-8af7-4fcf-b354-9d0ab4ade1a9 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:53.332299 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:53.372288 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60727e2-99b1-4b92-8dec-5135f4035b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:53.373651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60727e2-99b1-4b92-8dec-5135f4035b91 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:53.374265 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60727e2-99b1-4b92-8dec-5135f4035b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:53.375345 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60727e2-99b1-4b92-8dec-5135f4035b91 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:53.383900 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60727e2-99b1-4b92-8dec-5135f4035b91 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:53.385701 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60727e2-99b1-4b92-8dec-5135f4035b91 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:53.403604 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-a60727e2-99b1-4b92-8dec-5135f4035b91 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 03:09:53.403604 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:53.403604 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:53.403604 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:53.403604 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:53.403604 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:53.403604 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:53.403604 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:53.403604 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:53.403604 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:53.403604 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:53.403604 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:53.403604 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:09:53.403604 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:09:53.403604 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:09:53.403604 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:09:53.404791 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:53.404791 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:53.404791 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:09:53.404791 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:09:53.404791 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:09:53.404791 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:09:53.404791 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:53.404791 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:53.404791 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:53.404791 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:53.404791 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:53.404791 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:53.404791 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:53.404791 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:53.404791 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:53.404791 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:53.404791 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:53.404791 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:53.405855 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:53.405855 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:53.405855 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:09:53.405855 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:09:53.405855 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:09:53.405855 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:09:53.405855 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:09:53.405855 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:09:53.405855 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 03:09:53.405855 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:53.411337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 304/617] 10.222.0.50 () {56 vars in 1093 bytes} [Fri May 27 03:09:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:53.412537 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60727e2-99b1-4b92-8dec-5135f4035b91 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:53.482363 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:53.510342 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6fd068-ff7a-45d1-9894-6a5c37992abc None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:53.555771 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086140d2-1732-4ec3-a256-58ba894dc359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:53.556699 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086140d2-1732-4ec3-a256-58ba894dc359 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:53.557160 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086140d2-1732-4ec3-a256-58ba894dc359 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:53.557605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086140d2-1732-4ec3-a256-58ba894dc359 None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:53.558548 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086140d2-1732-4ec3-a256-58ba894dc359 None admin] RBAC: Adding query filter params (name=service) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:53.558994 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086140d2-1732-4ec3-a256-58ba894dc359 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:53.560557 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086140d2-1732-4ec3-a256-58ba894dc359 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:53.570218 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 314/618] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 03:09:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:53.575836 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60727e2-99b1-4b92-8dec-5135f4035b91 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:53.577035 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086140d2-1732-4ec3-a256-58ba894dc359 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:53.650446 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38d6f1-94ae-4e77-b692-644b519837e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:53.651413 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38d6f1-94ae-4e77-b692-644b519837e0 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:53.651841 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38d6f1-94ae-4e77-b692-644b519837e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:53.652275 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38d6f1-94ae-4e77-b692-644b519837e0 None admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:53.653350 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b38d6f1-94ae-4e77-b692-644b519837e0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:53.653807 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b38d6f1-94ae-4e77-b692-644b519837e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:53.655415 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b38d6f1-94ae-4e77-b692-644b519837e0 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:53.687620 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 305/619] 10.222.0.50 () {56 vars in 1133 bytes} [Fri May 27 03:09:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:53.692600 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38d6f1-94ae-4e77-b692-644b519837e0 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:53.725919 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:53.746721 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086140d2-1732-4ec3-a256-58ba894dc359 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:53.817773 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f198db-aa2f-4e57-8c4d-ca2f0b274a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:53.818599 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f198db-aa2f-4e57-8c4d-ca2f0b274a73 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:53.818885 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f198db-aa2f-4e57-8c4d-ca2f0b274a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:53.819158 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f198db-aa2f-4e57-8c4d-ca2f0b274a73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:53.846781 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38d6f1-94ae-4e77-b692-644b519837e0 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:53.847033 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f198db-aa2f-4e57-8c4d-ca2f0b274a73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:53.848999 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f198db-aa2f-4e57-8c4d-ca2f0b274a73 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:53.861424 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:53.885605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-c7f198db-aa2f-4e57-8c4d-ca2f0b274a73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:53.885605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:53.885605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:53.885605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:53.885605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:53.885605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:53.885605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:53.885605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:53.885605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:53.885605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:53.885605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:53.885605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:53.885605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:09:53.885605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:09:53.885605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:09:53.886739 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:09:53.886739 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:53.886739 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:53.886739 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:53.886739 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:53.886739 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:53.886739 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:53.886739 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:53.886739 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:53.886739 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:53.886739 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:53.886739 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:53.886739 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:53.886739 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:53.886739 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:53.886739 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:53.886739 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:53.887804 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:09:53.887804 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:09:53.887804 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:53.887804 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:53.888062 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c4faa-8956-41dc-8dae-3e774ff022b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:53.888062 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c4faa-8956-41dc-8dae-3e774ff022b7 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:53.888062 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c4faa-8956-41dc-8dae-3e774ff022b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:53.888062 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c4faa-8956-41dc-8dae-3e774ff022b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:53.892562 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 315/620] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:53.893801 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f198db-aa2f-4e57-8c4d-ca2f0b274a73 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:53.961021 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92c4faa-8956-41dc-8dae-3e774ff022b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:53.962558 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92c4faa-8956-41dc-8dae-3e774ff022b7 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:54.066759 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 306/621] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:09:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:09:54.072440 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c4faa-8956-41dc-8dae-3e774ff022b7 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:54.087460 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:54.126158 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f198db-aa2f-4e57-8c4d-ca2f0b274a73 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:54.162305 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b33ec4-1593-4598-a74f-9f08c687262e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:54.162983 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b33ec4-1593-4598-a74f-9f08c687262e None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:54.163267 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b33ec4-1593-4598-a74f-9f08c687262e None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:54.163547 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b33ec4-1593-4598-a74f-9f08c687262e None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:54.164278 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b33ec4-1593-4598-a74f-9f08c687262e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:54.164585 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b33ec4-1593-4598-a74f-9f08c687262e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:54.166182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b33ec4-1593-4598-a74f-9f08c687262e None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:54.179492 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 316/622] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:09:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:09:54.180100 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:54.185852 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b33ec4-1593-4598-a74f-9f08c687262e None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:54.259385 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c4faa-8956-41dc-8dae-3e774ff022b7 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:54.322423 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c04654-d908-402a-9971-fbe0db136c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:54.323321 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c04654-d908-402a-9971-fbe0db136c4f None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:54.323857 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c04654-d908-402a-9971-fbe0db136c4f None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:54.323857 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c04654-d908-402a-9971-fbe0db136c4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:54.337756 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c04654-d908-402a-9971-fbe0db136c4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:54.339221 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c04654-d908-402a-9971-fbe0db136c4f None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:54.353954 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b33ec4-1593-4598-a74f-9f08c687262e None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:54.384318 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-f2c04654-d908-402a-9971-fbe0db136c4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:54.384318 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:54.384318 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:54.384318 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:54.384318 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:54.384318 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:54.384318 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:54.384318 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:54.384318 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:54.384318 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:54.384318 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:54.384318 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:54.384318 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:09:54.384318 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:09:54.384318 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:09:54.385349 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:09:54.385349 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:54.385349 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:54.385349 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:54.385349 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:54.385349 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:54.385349 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:54.385349 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:54.385349 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:54.385349 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:54.385349 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:54.385349 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:54.385349 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:54.385349 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:54.385349 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:54.385349 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:54.385349 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:54.386232 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:09:54.386232 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:09:54.386232 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:54.386232 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:54.392789 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 307/623] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:54.399791 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c04654-d908-402a-9971-fbe0db136c4f None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:54.418807 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d14649-56fe-4050-9d2b-cb0cc7bd1c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:54.419718 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d14649-56fe-4050-9d2b-cb0cc7bd1c55 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:54.420322 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d14649-56fe-4050-9d2b-cb0cc7bd1c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:54.420431 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d14649-56fe-4050-9d2b-cb0cc7bd1c55 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:54.428425 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d14649-56fe-4050-9d2b-cb0cc7bd1c55 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:54.430759 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d14649-56fe-4050-9d2b-cb0cc7bd1c55 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:54.450376 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-e1d14649-56fe-4050-9d2b-cb0cc7bd1c55 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 03:09:54.450376 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:54.450376 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:54.450376 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:54.450376 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:54.450376 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:54.450376 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:54.450376 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:54.450376 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:54.450376 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:54.450376 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:54.450376 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:54.450376 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:09:54.450376 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:09:54.450376 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:09:54.450376 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:09:54.451392 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:54.451392 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:54.451392 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:09:54.451392 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:09:54.451392 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:09:54.451392 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:09:54.451392 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:54.451392 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:54.451392 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:54.451392 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:54.451392 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:54.451392 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:54.451392 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:54.451392 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:54.451392 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:54.451392 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:54.451392 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:54.451392 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:54.453788 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:54.453788 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:54.453788 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:09:54.453788 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:09:54.453788 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:09:54.453788 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:09:54.453788 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:09:54.453788 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:09:54.453788 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 03:09:54.453788 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:54.453788 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 317/624] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 03:09:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:09:54.454395 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95537181-118b-4665-a9e6-aa77350e3f38 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:54.455020 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95537181-118b-4665-a9e6-aa77350e3f38 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:54.455020 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95537181-118b-4665-a9e6-aa77350e3f38 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:54.456325 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 318/625] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:54.462561 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95537181-118b-4665-a9e6-aa77350e3f38 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:54.478464 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:54.520075 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c04654-d908-402a-9971-fbe0db136c4f None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:54.542541 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:54.560316 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c91dbf9-db25-4379-909a-4232f001651f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:54.561099 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91dbf9-db25-4379-909a-4232f001651f None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:54.561383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91dbf9-db25-4379-909a-4232f001651f None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:54.561668 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c91dbf9-db25-4379-909a-4232f001651f None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:54.562471 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c91dbf9-db25-4379-909a-4232f001651f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:54.562779 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c91dbf9-db25-4379-909a-4232f001651f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:54.564241 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c91dbf9-db25-4379-909a-4232f001651f None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:54.579334 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 308/626] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:09:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:54.582621 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee417e1-ea82-4551-aa26-8768001b4ea8 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:54.583362 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee417e1-ea82-4551-aa26-8768001b4ea8 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:54.583821 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee417e1-ea82-4551-aa26-8768001b4ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:54.602121 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-bee417e1-ea82-4551-aa26-8768001b4ea8 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:54.603632 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-bee417e1-ea82-4551-aa26-8768001b4ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:54.610058 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95537181-118b-4665-a9e6-aa77350e3f38 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:54.646228 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f05f04b-7d2b-4e90-9b11-cbe2e0e72c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:54.647224 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f05f04b-7d2b-4e90-9b11-cbe2e0e72c34 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:54.647746 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f05f04b-7d2b-4e90-9b11-cbe2e0e72c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:54.648196 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f05f04b-7d2b-4e90-9b11-cbe2e0e72c34 None admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:54.649285 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f05f04b-7d2b-4e90-9b11-cbe2e0e72c34 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:54.649771 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f05f04b-7d2b-4e90-9b11-cbe2e0e72c34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:54.651447 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f05f04b-7d2b-4e90-9b11-cbe2e0e72c34 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:54.675162 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 319/627] 10.222.0.50 () {56 vars in 1135 bytes} [Fri May 27 03:09:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:54.676429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f05f04b-7d2b-4e90-9b11-cbe2e0e72c34 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:54.698255 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 309/628] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:54.699454 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee417e1-ea82-4551-aa26-8768001b4ea8 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:54.772080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:54.804664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f05f04b-7d2b-4e90-9b11-cbe2e0e72c34 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:54.818526 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee417e1-ea82-4551-aa26-8768001b4ea8 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:54.818867 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:54.839567 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0417621-9d7e-4de8-8066-7b05be9f9f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:54.840379 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0417621-9d7e-4de8-8066-7b05be9f9f38 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:54.840662 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0417621-9d7e-4de8-8066-7b05be9f9f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:54.840934 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0417621-9d7e-4de8-8066-7b05be9f9f38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:54.847758 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0417621-9d7e-4de8-8066-7b05be9f9f38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:54.849501 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0417621-9d7e-4de8-8066-7b05be9f9f38 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:54.856108 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da55fe-82c1-4f2d-a3bb-9a467133f905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:54.856921 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da55fe-82c1-4f2d-a3bb-9a467133f905 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:54.857225 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da55fe-82c1-4f2d-a3bb-9a467133f905 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:54.857496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da55fe-82c1-4f2d-a3bb-9a467133f905 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:54.868630 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8da55fe-82c1-4f2d-a3bb-9a467133f905 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:54.870155 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8da55fe-82c1-4f2d-a3bb-9a467133f905 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:54.871208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-f0417621-9d7e-4de8-8066-7b05be9f9f38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:09:54.871208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:54.871208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:54.871208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:54.871208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:54.871208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:54.871208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:54.871208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:54.871208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:54.871208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:54.871208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:54.871208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:54.871208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:09:54.871208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:09:54.871208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:09:54.872111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:09:54.872111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:54.872111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:54.872111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:54.872111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:54.872111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:54.872111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:54.872111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:54.872111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:54.872111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:54.872111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:54.872111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:54.872111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:54.872111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:54.872111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:54.872111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:54.872111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:54.873504 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:09:54.873504 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:09:54.873504 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:09:54.873504 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:09:54.873504 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:09:54.873504 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:09:54.873504 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:09:54.873504 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:54.873504 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 320/629] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:09:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:54.873504 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0417621-9d7e-4de8-8066-7b05be9f9f38 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:54.892651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-a8da55fe-82c1-4f2d-a3bb-9a467133f905 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:54.892651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:54.892651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:54.892651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:54.892651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:54.892651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:54.892651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:54.892651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:54.892651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:54.892651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:54.892651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:54.892651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:54.892651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:09:54.892651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:09:54.892651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:09:54.893845 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:09:54.893845 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:54.893845 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:54.893845 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:54.893845 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:54.893845 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:54.893845 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:54.893845 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:54.893845 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:54.893845 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:54.893845 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:54.893845 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:54.893845 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:54.893845 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:54.893845 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:54.893845 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:54.893845 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:54.894812 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:09:54.894812 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:09:54.894812 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:54.894812 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:54.895741 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 310/630] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:54.897563 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443ceeb-65fa-4646-90f8-7360e4c13c94 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:54.898089 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443ceeb-65fa-4646-90f8-7360e4c13c94 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:54.898751 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d443ceeb-65fa-4646-90f8-7360e4c13c94 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:54.900444 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 311/631] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:54.901546 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443ceeb-65fa-4646-90f8-7360e4c13c94 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:54.988791 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0417621-9d7e-4de8-8066-7b05be9f9f38 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:55.010164 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d443ceeb-65fa-4646-90f8-7360e4c13c94 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:55.018884 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:55.026216 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da2832a-5579-4b5b-a829-cfc90431e38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:55.026954 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2832a-5579-4b5b-a829-cfc90431e38c None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:55.027236 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2832a-5579-4b5b-a829-cfc90431e38c None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:55.027531 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2832a-5579-4b5b-a829-cfc90431e38c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:55.038649 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da2832a-5579-4b5b-a829-cfc90431e38c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:55.046303 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-6da2832a-5579-4b5b-a829-cfc90431e38c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:09:55.046303 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:55.046303 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:55.046303 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:55.046303 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:55.046303 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:55.046303 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:55.046303 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:55.046303 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:55.046303 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:55.046303 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:55.046303 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:55.046303 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:55.046303 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:55.046303 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:55.046303 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:55.047238 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:55.047238 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:55.047238 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:55.047238 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:55.047238 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:55.047238 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:55.047238 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:55.047238 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:55.047238 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:55.047238 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:55.047238 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:55.047238 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:55.047238 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:55.047238 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:55.047238 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:55.047238 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:55.048048 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:55.048048 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:55.048048 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:55.048048 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:55.048048 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:55.048048 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:55.048048 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:55.048048 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:55.048048 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:55.048048 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:55.048048 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:09:55.048048 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:55.048048 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 321/632] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 03:09:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:55.049067 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da2832a-5579-4b5b-a829-cfc90431e38c None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:55.056460 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e849945f-043d-4213-acf6-39754ef1a463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:55.057166 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849945f-043d-4213-acf6-39754ef1a463 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:55.057427 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849945f-043d-4213-acf6-39754ef1a463 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:55.057708 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849945f-043d-4213-acf6-39754ef1a463 None admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:55.058476 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e849945f-043d-4213-acf6-39754ef1a463 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:55.058797 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e849945f-043d-4213-acf6-39754ef1a463 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:55.060227 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e849945f-043d-4213-acf6-39754ef1a463 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:55.072444 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 312/633] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 03:09:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:55.073730 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833f06b-0795-4297-b2ca-8696b2ab58c9 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:55.074101 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833f06b-0795-4297-b2ca-8696b2ab58c9 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:55.074384 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833f06b-0795-4297-b2ca-8696b2ab58c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:55.085555 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:55.091136 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-c833f06b-0795-4297-b2ca-8696b2ab58c9 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:55.092332 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-c833f06b-0795-4297-b2ca-8696b2ab58c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:55.152880 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da2832a-5579-4b5b-a829-cfc90431e38c None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:55.161844 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 313/634] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:55.162641 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c833f06b-0795-4297-b2ca-8696b2ab58c9 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:55.191976 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa7cd9-efb5-4024-b99d-00c253677638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:55.192638 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa7cd9-efb5-4024-b99d-00c253677638 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:55.192913 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa7cd9-efb5-4024-b99d-00c253677638 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:55.193216 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa7cd9-efb5-4024-b99d-00c253677638 None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:55.193931 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aa7cd9-efb5-4024-b99d-00c253677638 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:55.194230 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aa7cd9-efb5-4024-b99d-00c253677638 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:55.195646 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aa7cd9-efb5-4024-b99d-00c253677638 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:55.207659 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 322/635] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:09:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:55.208939 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b9de2-7ab4-4273-8111-c0d46c71a74f None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:55.209245 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b9de2-7ab4-4273-8111-c0d46c71a74f None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:55.209605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6b9de2-7ab4-4273-8111-c0d46c71a74f None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:55.210863 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 323/636] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:55.211399 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6b9de2-7ab4-4273-8111-c0d46c71a74f None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:55.261279 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:55.268192 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c833f06b-0795-4297-b2ca-8696b2ab58c9 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:55.303259 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:55.306810 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4518c8e-3c36-49b4-add6-021fec619a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:55.307502 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4518c8e-3c36-49b4-add6-021fec619a97 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:55.307786 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4518c8e-3c36-49b4-add6-021fec619a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:55.308061 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4518c8e-3c36-49b4-add6-021fec619a97 None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:55.308849 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4518c8e-3c36-49b4-add6-021fec619a97 None admin] RBAC: Adding query filter params (name=service) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:55.309193 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4518c8e-3c36-49b4-add6-021fec619a97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:55.310562 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4518c8e-3c36-49b4-add6-021fec619a97 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:55.311286 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6b9de2-7ab4-4273-8111-c0d46c71a74f None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:55.318923 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 314/637] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 03:09:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:55.319710 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4518c8e-3c36-49b4-add6-021fec619a97 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:55.349082 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbabc3f7-f1d0-47f7-8c53-b70b65c24b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:55.349747 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbabc3f7-f1d0-47f7-8c53-b70b65c24b79 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:55.350034 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbabc3f7-f1d0-47f7-8c53-b70b65c24b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:55.350307 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbabc3f7-f1d0-47f7-8c53-b70b65c24b79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:55.351132 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbabc3f7-f1d0-47f7-8c53-b70b65c24b79 None admin] RBAC: Adding query filter params (role.id=3451aa51e64642768b2228126d20fa03, scope.project.id=f8398941252346468483ea0c863122f4, user.id=1a699988f08a4001b3b7d764f01f4fc5) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:55.351441 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbabc3f7-f1d0-47f7-8c53-b70b65c24b79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:55.352893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbabc3f7-f1d0-47f7-8c53-b70b65c24b79 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:55.358724 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 324/638] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:09:55 2022] GET /identity/v3/role_assignments?user.id=1a699988f08a4001b3b7d764f01f4fc5&scope.project.id=f8398941252346468483ea0c863122f4&role.id=3451aa51e64642768b2228126d20fa03 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:55.359503 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbabc3f7-f1d0-47f7-8c53-b70b65c24b79 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:55.424831 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4518c8e-3c36-49b4-add6-021fec619a97 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:55.462881 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f020eda-21ab-4fa6-ae96-5f0455f38919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:55.463725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f020eda-21ab-4fa6-ae96-5f0455f38919 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:55.464016 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f020eda-21ab-4fa6-ae96-5f0455f38919 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:55.464295 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f020eda-21ab-4fa6-ae96-5f0455f38919 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:55.477276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f020eda-21ab-4fa6-ae96-5f0455f38919 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:55.479664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbabc3f7-f1d0-47f7-8c53-b70b65c24b79 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:55.485600 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-9f020eda-21ab-4fa6-ae96-5f0455f38919 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:09:55.485600 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:55.485600 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:55.485600 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:55.485600 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:55.485600 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:55.485600 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:55.485600 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:55.485600 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:55.485600 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:55.485600 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:55.485600 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:55.485600 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:55.485600 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:55.485600 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:55.485600 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:55.486704 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:55.486704 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:55.486704 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:55.486704 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:55.486704 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:55.486704 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:55.486704 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:55.486704 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:55.486704 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:55.486704 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:55.486704 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:55.486704 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:55.486704 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:55.486704 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:55.486704 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:55.486704 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:55.492773 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:55.492773 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:55.492773 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:55.492773 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:55.492773 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:55.492773 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:55.492773 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:55.492773 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:55.492773 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:55.492773 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:55.492773 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:09:55.492773 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:55.492773 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 315/639] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 03:09:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:55.492773 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c688fe92-f2fb-43d9-a273-3ab677dcde8f None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:55.493601 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c688fe92-f2fb-43d9-a273-3ab677dcde8f None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:55.493601 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c688fe92-f2fb-43d9-a273-3ab677dcde8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:55.509499 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-c688fe92-f2fb-43d9-a273-3ab677dcde8f None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:55.510830 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-c688fe92-f2fb-43d9-a273-3ab677dcde8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:55.524757 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db89ebfc-d6a0-47c4-b52f-766264c8a6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:55.525862 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89ebfc-d6a0-47c4-b52f-766264c8a6d7 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:55.526137 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89ebfc-d6a0-47c4-b52f-766264c8a6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:55.526408 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89ebfc-d6a0-47c4-b52f-766264c8a6d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:55.526979 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:55.534855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db89ebfc-d6a0-47c4-b52f-766264c8a6d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:55.536568 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db89ebfc-d6a0-47c4-b52f-766264c8a6d7 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:55.552436 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-db89ebfc-d6a0-47c4-b52f-766264c8a6d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:09:55.552436 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:55.552436 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:55.552436 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:55.552436 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:55.552436 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:55.552436 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:55.552436 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:55.552436 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:55.552436 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:55.552436 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:55.552436 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:55.552436 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:09:55.552436 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:09:55.552436 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:09:55.553368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:09:55.553368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:55.553368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:55.553368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:55.553368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:55.553368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:55.553368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:55.553368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:55.553368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:55.553368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:55.553368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:55.553368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:55.553368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:55.553368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:55.553368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:55.553368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:55.553368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:55.554234 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:09:55.554234 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:09:55.554234 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:09:55.554234 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:09:55.554234 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:09:55.554234 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:09:55.554234 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:09:55.554234 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:55.554234 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 325/640] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:09:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:55.554887 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db89ebfc-d6a0-47c4-b52f-766264c8a6d7 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:55.576965 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:55.592982 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 316/641] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:55.592982 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c688fe92-f2fb-43d9-a273-3ab677dcde8f None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:55.677731 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db89ebfc-d6a0-47c4-b52f-766264c8a6d7 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:55.700740 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c688fe92-f2fb-43d9-a273-3ab677dcde8f None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:55.713781 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccaf6bd-74c0-48ab-bb43-917386b0e6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:55.714678 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccaf6bd-74c0-48ab-bb43-917386b0e6ac None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:55.715034 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccaf6bd-74c0-48ab-bb43-917386b0e6ac None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:55.715314 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccaf6bd-74c0-48ab-bb43-917386b0e6ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:55.727021 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccaf6bd-74c0-48ab-bb43-917386b0e6ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:55.728502 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccaf6bd-74c0-48ab-bb43-917386b0e6ac None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:55.736488 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fcda9a-5f41-4674-998b-2efedf587817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:55.737186 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fcda9a-5f41-4674-998b-2efedf587817 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:55.737465 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fcda9a-5f41-4674-998b-2efedf587817 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:55.737734 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fcda9a-5f41-4674-998b-2efedf587817 None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:55.738537 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fcda9a-5f41-4674-998b-2efedf587817 None admin] RBAC: Adding query filter params (name=service) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:55.738838 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fcda9a-5f41-4674-998b-2efedf587817 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:55.740351 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fcda9a-5f41-4674-998b-2efedf587817 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:55.746103 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-0ccaf6bd-74c0-48ab-bb43-917386b0e6ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:55.746103 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:55.746103 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:55.746103 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:55.746103 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:55.746103 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:55.746103 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:55.746103 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:55.746103 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:55.746103 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:55.746103 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:55.746103 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:55.746103 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:09:55.746103 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:09:55.746103 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:09:55.747011 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:09:55.747011 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:55.747011 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:55.747011 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:55.747011 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:55.747011 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:55.747011 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:55.747011 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:55.747011 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:55.747011 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:55.747011 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:55.747011 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:55.747011 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:55.747011 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:55.747011 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:55.747011 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:55.747011 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:55.747914 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:09:55.747914 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:09:55.747914 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:55.747914 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:55.747914 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 326/642] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:55.748214 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccaf6bd-74c0-48ab-bb43-917386b0e6ac None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:55.754866 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 317/643] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 03:09:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:55.755551 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fcda9a-5f41-4674-998b-2efedf587817 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:55.773089 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:55.826684 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:55.859895 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccaf6bd-74c0-48ab-bb43-917386b0e6ac None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:55.868625 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fcda9a-5f41-4674-998b-2efedf587817 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:55.895645 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7b73d-0cb3-4332-82c7-ec487dd96066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:55.896325 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7b73d-0cb3-4332-82c7-ec487dd96066 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:55.896609 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7b73d-0cb3-4332-82c7-ec487dd96066 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:55.896883 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7b73d-0cb3-4332-82c7-ec487dd96066 None admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:55.897712 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7b73d-0cb3-4332-82c7-ec487dd96066 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:55.898018 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7b73d-0cb3-4332-82c7-ec487dd96066 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:55.899468 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7b73d-0cb3-4332-82c7-ec487dd96066 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:55.904221 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93bd116-0adf-4c88-b6c4-0513660c2652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:55.905058 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93bd116-0adf-4c88-b6c4-0513660c2652 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:55.905341 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93bd116-0adf-4c88-b6c4-0513660c2652 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:55.905611 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93bd116-0adf-4c88-b6c4-0513660c2652 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:55.916784 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 327/644] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 03:09:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:55.916784 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7b73d-0cb3-4332-82c7-ec487dd96066 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:55.923367 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93bd116-0adf-4c88-b6c4-0513660c2652 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:55.931366 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-b93bd116-0adf-4c88-b6c4-0513660c2652 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:09:55.931366 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:55.931366 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:55.931366 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:55.931366 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:55.931366 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:55.931366 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:55.931366 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:55.931366 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:55.931366 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:55.931366 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:55.931366 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:55.931366 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:55.931366 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:55.931366 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:55.931366 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:55.932508 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:55.932508 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:55.932508 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:55.932508 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:55.932508 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:55.932508 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:55.932508 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:55.932508 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:55.932508 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:55.932508 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:55.932508 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:55.932508 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:55.932508 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:55.932508 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:55.932508 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:55.932508 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:55.938434 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:55.938434 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:55.938434 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:55.938434 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:55.938434 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:55.938434 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:55.938434 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:55.938434 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:55.938434 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:55.938434 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:55.938434 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:09:55.938434 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:55.938434 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 318/645] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 03:09:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:55.938434 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93bd116-0adf-4c88-b6c4-0513660c2652 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:56.011257 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:56.024455 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7b73d-0cb3-4332-82c7-ec487dd96066 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:56.053646 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93bd116-0adf-4c88-b6c4-0513660c2652 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:56.054602 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:56.061672 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d81bc00-aa98-4730-bfb7-d12638e292e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:56.062126 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81bc00-aa98-4730-bfb7-d12638e292e8 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:56.062402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81bc00-aa98-4730-bfb7-d12638e292e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:56.062673 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81bc00-aa98-4730-bfb7-d12638e292e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:56.063392 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d81bc00-aa98-4730-bfb7-d12638e292e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:56.063698 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d81bc00-aa98-4730-bfb7-d12638e292e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:56.065114 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d81bc00-aa98-4730-bfb7-d12638e292e8 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:56.078018 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 328/646] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:09:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:56.079039 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d81bc00-aa98-4730-bfb7-d12638e292e8 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:56.095845 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26582b7d-1cf0-463d-a709-0a035d42b7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:56.097275 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26582b7d-1cf0-463d-a709-0a035d42b7e6 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:56.097533 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26582b7d-1cf0-463d-a709-0a035d42b7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:56.098124 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26582b7d-1cf0-463d-a709-0a035d42b7e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:56.109416 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26582b7d-1cf0-463d-a709-0a035d42b7e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:56.111079 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26582b7d-1cf0-463d-a709-0a035d42b7e6 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:56.129200 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-26582b7d-1cf0-463d-a709-0a035d42b7e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:56.129200 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:56.129200 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:56.129200 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:56.129200 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:56.129200 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:56.129200 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:56.129200 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:56.129200 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:56.129200 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:56.129200 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:56.129200 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:56.129200 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:09:56.129200 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:09:56.129200 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:09:56.130325 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:09:56.130325 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:56.130325 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:56.130325 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:56.130325 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:56.130325 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:56.130325 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:56.130325 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:56.130325 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:56.130325 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:56.130325 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:56.130325 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:56.130325 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:56.130325 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:56.130325 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:56.130325 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:56.130325 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:56.131277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:09:56.131277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:09:56.131277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:56.131277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:56.131277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 319/647] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:56.131690 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26582b7d-1cf0-463d-a709-0a035d42b7e6 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:56.192537 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d81bc00-aa98-4730-bfb7-d12638e292e8 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:56.228334 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41477263-78c2-4e59-8898-c8142f44fc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:56.229367 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41477263-78c2-4e59-8898-c8142f44fc61 None admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:56.229716 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41477263-78c2-4e59-8898-c8142f44fc61 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:56.229996 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41477263-78c2-4e59-8898-c8142f44fc61 None admin] PATH_INFO: `/v3/projects/f8398941252346468483ea0c863122f4/users/3de53e2f0fba4e0fac70887067c1963d/roles/3451aa51e64642768b2228126d20fa03` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:56.236425 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26582b7d-1cf0-463d-a709-0a035d42b7e6 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:56.245861 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41477263-78c2-4e59-8898-c8142f44fc61 None admin] RBAC: Authorizing `identity:create_grant(project_id=f8398941252346468483ea0c863122f4, user_id=3de53e2f0fba4e0fac70887067c1963d, role_id=3451aa51e64642768b2228126d20fa03)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:56.247346 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41477263-78c2-4e59-8898-c8142f44fc61 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:56.260201 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:56.269040 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 329/648] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 03:09:56 2022] PUT /identity/v3/projects/f8398941252346468483ea0c863122f4/users/3de53e2f0fba4e0fac70887067c1963d/roles/3451aa51e64642768b2228126d20fa03 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:09:56.269040 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41477263-78c2-4e59-8898-c8142f44fc61 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:56.286604 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a18254-1bc0-493f-ac37-ada67ca231da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:56.287286 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a18254-1bc0-493f-ac37-ada67ca231da None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:56.287610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a18254-1bc0-493f-ac37-ada67ca231da None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:56.287882 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a18254-1bc0-493f-ac37-ada67ca231da None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:56.288814 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a18254-1bc0-493f-ac37-ada67ca231da None admin] RBAC: Adding query filter params (name=service) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:56.289165 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a18254-1bc0-493f-ac37-ada67ca231da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:56.290801 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a18254-1bc0-493f-ac37-ada67ca231da None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:56.304193 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 320/649] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 03:09:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:56.305230 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a18254-1bc0-493f-ac37-ada67ca231da None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:56.323869 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:56.401812 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41477263-78c2-4e59-8898-c8142f44fc61 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:56.437931 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a18254-1bc0-493f-ac37-ada67ca231da None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:56.451688 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36afd2a8-a057-422c-ae9a-ccd4876de2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:56.452817 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36afd2a8-a057-422c-ae9a-ccd4876de2d1 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:56.453417 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36afd2a8-a057-422c-ae9a-ccd4876de2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:56.453836 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36afd2a8-a057-422c-ae9a-ccd4876de2d1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:56.464483 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36afd2a8-a057-422c-ae9a-ccd4876de2d1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:56.466959 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36afd2a8-a057-422c-ae9a-ccd4876de2d1 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:56.477382 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9229436-0af2-465b-9c3b-4c3243c7858d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:56.478217 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9229436-0af2-465b-9c3b-4c3243c7858d None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:56.478596 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9229436-0af2-465b-9c3b-4c3243c7858d None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:56.479037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9229436-0af2-465b-9c3b-4c3243c7858d None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:56.479873 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9229436-0af2-465b-9c3b-4c3243c7858d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:56.480279 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9229436-0af2-465b-9c3b-4c3243c7858d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:56.481857 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9229436-0af2-465b-9c3b-4c3243c7858d None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:56.486620 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-36afd2a8-a057-422c-ae9a-ccd4876de2d1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 27 03:09:56.486620 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:56.486620 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:56.486620 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:56.486620 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:56.486620 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:56.486620 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:56.486620 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:56.486620 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:56.486620 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:56.486620 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:56.486620 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:56.486620 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:09:56.486620 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:09:56.486620 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:09:56.486620 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:09:56.487629 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:56.487629 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:56.487629 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:09:56.487629 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:09:56.487629 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:09:56.487629 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:09:56.487629 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:56.487629 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:56.487629 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:56.487629 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:56.487629 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:56.487629 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:56.487629 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:56.487629 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:56.487629 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:56.487629 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:56.487629 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:56.487629 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:56.488773 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:56.488773 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:56.488773 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:09:56.488773 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:09:56.488773 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:09:56.488773 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:09:56.488773 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:09:56.488773 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:09:56.488773 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 27 03:09:56.488773 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:56.491060 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 330/650] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:09:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:56.492039 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36afd2a8-a057-422c-ae9a-ccd4876de2d1 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:56.502644 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 321/651] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:09:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:56.503245 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9229436-0af2-465b-9c3b-4c3243c7858d None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:56.562613 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:56.612918 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:56.616417 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36afd2a8-a057-422c-ae9a-ccd4876de2d1 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:56.622737 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9229436-0af2-465b-9c3b-4c3243c7858d None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:56.655082 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2bae77-456d-44b3-8a22-8298d9a2ac9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:56.655806 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2bae77-456d-44b3-8a22-8298d9a2ac9a None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:56.656083 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2bae77-456d-44b3-8a22-8298d9a2ac9a None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:56.656359 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2bae77-456d-44b3-8a22-8298d9a2ac9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:56.659707 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61215df8-bd85-455c-9193-328fcd396dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:56.660598 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61215df8-bd85-455c-9193-328fcd396dbb None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:56.661079 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61215df8-bd85-455c-9193-328fcd396dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:56.661491 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61215df8-bd85-455c-9193-328fcd396dbb None admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:56.662470 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61215df8-bd85-455c-9193-328fcd396dbb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:56.662935 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61215df8-bd85-455c-9193-328fcd396dbb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:56.664521 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61215df8-bd85-455c-9193-328fcd396dbb None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:56.668073 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2bae77-456d-44b3-8a22-8298d9a2ac9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:56.669458 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2bae77-456d-44b3-8a22-8298d9a2ac9a None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:56.686122 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 322/652] 10.222.0.50 () {56 vars in 1137 bytes} [Fri May 27 03:09:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:56.687216 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61215df8-bd85-455c-9193-328fcd396dbb None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:56.695831 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-9b2bae77-456d-44b3-8a22-8298d9a2ac9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:56.695831 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:56.695831 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:56.695831 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:56.695831 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:56.695831 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:56.695831 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:56.695831 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:56.695831 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:56.695831 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:56.695831 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:56.695831 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:56.695831 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:09:56.695831 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:09:56.695831 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:09:56.696910 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:09:56.696910 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:56.696910 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:56.696910 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:56.696910 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:56.696910 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:56.696910 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:56.696910 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:56.696910 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:56.696910 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:56.696910 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:56.696910 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:56.696910 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:56.696910 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:56.696910 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:56.696910 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:56.696910 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:56.697911 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:09:56.697911 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:09:56.697911 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:56.697911 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:56.699794 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 331/653] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:56.700783 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2bae77-456d-44b3-8a22-8298d9a2ac9a None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:56.828406 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2bae77-456d-44b3-8a22-8298d9a2ac9a None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:56.832062 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61215df8-bd85-455c-9193-328fcd396dbb None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:56.833932 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:56.866029 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc6daab-1802-49c8-ae05-c8b5f68bfd1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:56.866662 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc6daab-1802-49c8-ae05-c8b5f68bfd1e None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:56.866937 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc6daab-1802-49c8-ae05-c8b5f68bfd1e None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:56.867206 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc6daab-1802-49c8-ae05-c8b5f68bfd1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:56.886277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b282307-5f98-44e2-b185-0a2974fbcc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:56.887535 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b282307-5f98-44e2-b185-0a2974fbcc8f None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:56.888042 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b282307-5f98-44e2-b185-0a2974fbcc8f None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:56.888578 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b282307-5f98-44e2-b185-0a2974fbcc8f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:56.891617 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:56.898176 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b282307-5f98-44e2-b185-0a2974fbcc8f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:56.900542 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b282307-5f98-44e2-b185-0a2974fbcc8f None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:56.917075 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-3b282307-5f98-44e2-b185-0a2974fbcc8f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 27 03:09:56.917075 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:56.917075 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:56.917075 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:56.917075 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:56.917075 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:56.917075 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:56.917075 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:56.917075 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:56.917075 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:56.917075 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:56.917075 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:56.917075 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:09:56.917075 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:09:56.917075 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:09:56.917075 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:09:56.918025 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:56.918025 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:56.918025 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:09:56.918025 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:09:56.918025 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:09:56.918025 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:09:56.918025 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:56.918025 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:56.918025 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:56.918025 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:56.918025 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:56.918025 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:56.918025 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:56.918025 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:56.918025 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:56.918025 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:56.918025 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:56.918025 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:56.918961 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:56.918961 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:56.918961 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:09:56.918961 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:09:56.918961 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:09:56.918961 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:09:56.918961 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:09:56.918961 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:09:56.918961 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 27 03:09:56.918961 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:56.921689 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 323/654] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:09:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:56.922767 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b282307-5f98-44e2-b185-0a2974fbcc8f None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:56.943649 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc6daab-1802-49c8-ae05-c8b5f68bfd1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:56.945162 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc6daab-1802-49c8-ae05-c8b5f68bfd1e None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:57.036218 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b282307-5f98-44e2-b185-0a2974fbcc8f None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:57.056143 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 332/655] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:09:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:09:57.057245 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc6daab-1802-49c8-ae05-c8b5f68bfd1e None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:57.075293 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ae9fa-9ee2-4fae-8437-04a7201d0e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:57.076421 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ae9fa-9ee2-4fae-8437-04a7201d0e04 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:57.077107 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ae9fa-9ee2-4fae-8437-04a7201d0e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:57.077582 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ae9fa-9ee2-4fae-8437-04a7201d0e04 None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:57.078511 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ae9fa-9ee2-4fae-8437-04a7201d0e04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:57.079013 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ae9fa-9ee2-4fae-8437-04a7201d0e04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:57.080687 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ae9fa-9ee2-4fae-8437-04a7201d0e04 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:57.097231 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 324/656] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:09:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:57.098569 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ae9fa-9ee2-4fae-8437-04a7201d0e04 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:57.112804 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:57.129298 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:57.165799 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc6daab-1802-49c8-ae05-c8b5f68bfd1e None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:57.202229 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176aa1a4-bb2c-4fd1-8851-b360c4f2f013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:57.202907 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176aa1a4-bb2c-4fd1-8851-b360c4f2f013 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:57.203185 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176aa1a4-bb2c-4fd1-8851-b360c4f2f013 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:57.203455 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176aa1a4-bb2c-4fd1-8851-b360c4f2f013 None admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:57.204287 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176aa1a4-bb2c-4fd1-8851-b360c4f2f013 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:57.204592 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176aa1a4-bb2c-4fd1-8851-b360c4f2f013 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:57.206180 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176aa1a4-bb2c-4fd1-8851-b360c4f2f013 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:57.212717 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ae9fa-9ee2-4fae-8437-04a7201d0e04 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:57.224990 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 333/657] 10.222.0.50 () {56 vars in 1137 bytes} [Fri May 27 03:09:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:57.225861 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176aa1a4-bb2c-4fd1-8851-b360c4f2f013 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:57.249947 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e1b7a9-2f4c-4984-8805-9a0e2664b282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:57.251214 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1b7a9-2f4c-4984-8805-9a0e2664b282 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:57.251739 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1b7a9-2f4c-4984-8805-9a0e2664b282 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:57.252208 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1b7a9-2f4c-4984-8805-9a0e2664b282 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:57.264430 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e1b7a9-2f4c-4984-8805-9a0e2664b282 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:57.266248 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e1b7a9-2f4c-4984-8805-9a0e2664b282 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:57.292414 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-11e1b7a9-2f4c-4984-8805-9a0e2664b282 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:57.292414 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:57.292414 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:57.292414 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:57.292414 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:57.292414 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:57.292414 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:57.292414 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:57.292414 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:57.292414 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:57.292414 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:57.292414 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:57.292414 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:09:57.292414 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:09:57.292414 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:09:57.293406 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:09:57.293406 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:57.293406 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:57.293406 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:57.293406 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:57.293406 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:57.293406 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:57.293406 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:57.293406 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:57.293406 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:57.293406 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:57.293406 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:57.293406 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:57.293406 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:57.293406 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:57.293406 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:57.293406 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:57.294290 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:09:57.294290 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:09:57.294290 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:57.294290 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:57.294290 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 325/658] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:57.294561 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e1b7a9-2f4c-4984-8805-9a0e2664b282 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:57.352360 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176aa1a4-bb2c-4fd1-8851-b360c4f2f013 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:57.364414 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:57.390638 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ef276-f10c-4e0f-9a56-0337daa7b5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:57.391541 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ef276-f10c-4e0f-9a56-0337daa7b5bd None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:57.391897 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ef276-f10c-4e0f-9a56-0337daa7b5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:57.392226 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271ef276-f10c-4e0f-9a56-0337daa7b5bd None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:57.396971 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:57.401037 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271ef276-f10c-4e0f-9a56-0337daa7b5bd None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:57.402628 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271ef276-f10c-4e0f-9a56-0337daa7b5bd None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:57.418231 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-271ef276-f10c-4e0f-9a56-0337daa7b5bd None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 27 03:09:57.418231 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:57.418231 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:57.418231 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:57.418231 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:57.418231 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:57.418231 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:57.418231 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:57.418231 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:57.418231 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:57.418231 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:57.418231 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:57.418231 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:09:57.418231 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:09:57.418231 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:09:57.418231 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:09:57.419210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:57.419210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:57.419210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:09:57.419210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:09:57.419210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:09:57.419210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:09:57.419210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:57.419210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:57.419210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:57.419210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:57.419210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:57.419210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:57.419210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:57.419210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:57.419210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:57.419210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:57.419210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:57.419210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:57.420143 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:57.420143 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:57.420143 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:09:57.420143 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:09:57.420143 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:09:57.420143 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:09:57.420143 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:09:57.420143 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:09:57.420143 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 27 03:09:57.420143 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:57.424342 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 334/659] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 03:09:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:57.425297 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ef276-f10c-4e0f-9a56-0337daa7b5bd None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:57.431462 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e1b7a9-2f4c-4984-8805-9a0e2664b282 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:57.482096 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047438fb-ea71-44dc-a2b5-80d5c9ded405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:57.484043 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047438fb-ea71-44dc-a2b5-80d5c9ded405 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:57.484733 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047438fb-ea71-44dc-a2b5-80d5c9ded405 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:57.485214 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047438fb-ea71-44dc-a2b5-80d5c9ded405 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:57.552107 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271ef276-f10c-4e0f-9a56-0337daa7b5bd None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:57.573048 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047438fb-ea71-44dc-a2b5-80d5c9ded405 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:57.575461 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047438fb-ea71-44dc-a2b5-80d5c9ded405 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:57.592783 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f9b60-631b-4e04-bea6-0afdc0b53c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:57.593610 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f9b60-631b-4e04-bea6-0afdc0b53c7e None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:57.593984 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f9b60-631b-4e04-bea6-0afdc0b53c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:57.594363 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f9b60-631b-4e04-bea6-0afdc0b53c7e None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:57.595183 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9f9b60-631b-4e04-bea6-0afdc0b53c7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:57.595581 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9f9b60-631b-4e04-bea6-0afdc0b53c7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:57.597089 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9f9b60-631b-4e04-bea6-0afdc0b53c7e None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:57.614701 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 335/660] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:09:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:57.615770 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f9b60-631b-4e04-bea6-0afdc0b53c7e None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:57.647657 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:57.672613 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:57.700423 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 326/661] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:09:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:09:57.701288 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047438fb-ea71-44dc-a2b5-80d5c9ded405 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:57.728351 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f9b60-631b-4e04-bea6-0afdc0b53c7e None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:57.763615 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c929f5-9441-4133-a2e6-bd39bd3f2d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:57.764774 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c929f5-9441-4133-a2e6-bd39bd3f2d79 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:57.765320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c929f5-9441-4133-a2e6-bd39bd3f2d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:57.765723 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c929f5-9441-4133-a2e6-bd39bd3f2d79 None admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:57.766618 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c929f5-9441-4133-a2e6-bd39bd3f2d79 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:57.766932 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c929f5-9441-4133-a2e6-bd39bd3f2d79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:57.768475 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c929f5-9441-4133-a2e6-bd39bd3f2d79 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:57.792192 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 336/662] 10.222.0.50 () {56 vars in 1139 bytes} [Fri May 27 03:09:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:57.792192 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29e94e-cb7b-480c-bbf8-506508d65fd5 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:57.792192 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29e94e-cb7b-480c-bbf8-506508d65fd5 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:57.792192 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29e94e-cb7b-480c-bbf8-506508d65fd5 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:57.792192 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 337/663] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:57.792192 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b29e94e-cb7b-480c-bbf8-506508d65fd5 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:57.812523 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047438fb-ea71-44dc-a2b5-80d5c9ded405 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:57.848773 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed55de5-d26c-4da0-8587-66684c710d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:57.850008 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed55de5-d26c-4da0-8587-66684c710d17 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:57.850293 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed55de5-d26c-4da0-8587-66684c710d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:57.850566 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed55de5-d26c-4da0-8587-66684c710d17 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:57.857662 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed55de5-d26c-4da0-8587-66684c710d17 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:57.859126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed55de5-d26c-4da0-8587-66684c710d17 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:57.872821 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-2ed55de5-d26c-4da0-8587-66684c710d17 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:09:57.872821 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:57.872821 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:57.872821 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:57.872821 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:57.872821 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:57.872821 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:57.872821 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:57.872821 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:57.872821 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:57.872821 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:57.872821 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:57.872821 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:09:57.872821 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:09:57.872821 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:09:57.873777 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:09:57.873777 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:57.873777 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:57.873777 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:57.873777 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:57.873777 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:57.873777 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:57.873777 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:57.873777 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:57.873777 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:57.873777 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:57.873777 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:57.873777 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:57.873777 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:57.873777 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:57.873777 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:57.873777 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:57.874702 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:09:57.874702 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:09:57.874702 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:09:57.874702 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:09:57.874702 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:09:57.874702 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:09:57.874702 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:09:57.874702 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:57.878651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 327/664] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:09:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:57.879744 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed55de5-d26c-4da0-8587-66684c710d17 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:57.887924 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:57.901279 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b29e94e-cb7b-480c-bbf8-506508d65fd5 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:57.917997 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:57.941711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34640bb-0602-45f9-ae45-8f9e61f54468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:57.942475 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34640bb-0602-45f9-ae45-8f9e61f54468 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:57.943069 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34640bb-0602-45f9-ae45-8f9e61f54468 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:57.943350 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34640bb-0602-45f9-ae45-8f9e61f54468 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:57.955620 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34640bb-0602-45f9-ae45-8f9e61f54468 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:57.957222 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34640bb-0602-45f9-ae45-8f9e61f54468 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:57.975701 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-c34640bb-0602-45f9-ae45-8f9e61f54468 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:57.975701 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:57.975701 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:57.975701 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:57.975701 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:57.975701 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:57.975701 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:57.975701 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:57.975701 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:57.975701 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:57.975701 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:57.975701 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:57.975701 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:09:57.975701 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:09:57.975701 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:09:57.976843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:09:57.976843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:57.976843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:57.976843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:57.976843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:57.976843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:57.976843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:57.976843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:57.976843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:57.976843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:57.976843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:57.976843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:57.976843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:57.976843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:57.976843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:57.976843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:57.976843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:57.978001 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:09:57.978001 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:09:57.978001 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:57.978001 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:57.981398 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 338/665] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:57.982741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88220ffc-9482-42bc-a734-f3460b9b5d85 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:57.983061 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88220ffc-9482-42bc-a734-f3460b9b5d85 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:57.983310 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88220ffc-9482-42bc-a734-f3460b9b5d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:57.992625 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed55de5-d26c-4da0-8587-66684c710d17 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:58.000149 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-88220ffc-9482-42bc-a734-f3460b9b5d85 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:58.001378 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-88220ffc-9482-42bc-a734-f3460b9b5d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:58.029633 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f22983-128c-46e1-b27d-1c4773c82d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:58.030234 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f22983-128c-46e1-b27d-1c4773c82d5b None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:58.030493 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f22983-128c-46e1-b27d-1c4773c82d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:58.030765 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f22983-128c-46e1-b27d-1c4773c82d5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:58.074554 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 339/666] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:58.075530 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88220ffc-9482-42bc-a734-f3460b9b5d85 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:58.116088 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f22983-128c-46e1-b27d-1c4773c82d5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:58.117715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f22983-128c-46e1-b27d-1c4773c82d5b None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:58.154719 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:58.155889 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:58.187508 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88220ffc-9482-42bc-a734-f3460b9b5d85 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:58.223083 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36380866-2b8a-4eb7-864e-816a11374683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:58.223752 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36380866-2b8a-4eb7-864e-816a11374683 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:58.224195 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36380866-2b8a-4eb7-864e-816a11374683 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:58.224315 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36380866-2b8a-4eb7-864e-816a11374683 None admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:58.225098 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36380866-2b8a-4eb7-864e-816a11374683 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:58.225408 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36380866-2b8a-4eb7-864e-816a11374683 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:58.226852 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36380866-2b8a-4eb7-864e-816a11374683 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:58.239083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 328/667] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:09:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:09:58.239083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f22983-128c-46e1-b27d-1c4773c82d5b None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:58.244250 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 340/668] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 03:09:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:58.245937 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36380866-2b8a-4eb7-864e-816a11374683 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:58.350041 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f22983-128c-46e1-b27d-1c4773c82d5b None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:58.358251 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36380866-2b8a-4eb7-864e-816a11374683 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:58.388537 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d982e-3805-4fb4-ae16-00efb8ca427a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:58.389333 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d982e-3805-4fb4-ae16-00efb8ca427a None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:58.389682 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d982e-3805-4fb4-ae16-00efb8ca427a None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:58.389970 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1d982e-3805-4fb4-ae16-00efb8ca427a None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:58.390438 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:58.390689 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d982e-3805-4fb4-ae16-00efb8ca427a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:58.391051 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d982e-3805-4fb4-ae16-00efb8ca427a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:58.392414 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1d982e-3805-4fb4-ae16-00efb8ca427a None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:58.395575 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:58.399512 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d53725-92ed-4350-be9b-96bb6aee406a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:58.400494 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d53725-92ed-4350-be9b-96bb6aee406a None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:58.400872 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d53725-92ed-4350-be9b-96bb6aee406a None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:58.401270 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d53725-92ed-4350-be9b-96bb6aee406a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:58.412780 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 329/669] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:09:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:58.412780 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d982e-3805-4fb4-ae16-00efb8ca427a None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:58.417190 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d53725-92ed-4350-be9b-96bb6aee406a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:58.424804 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-83d53725-92ed-4350-be9b-96bb6aee406a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:09:58.424804 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:58.424804 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:58.424804 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:58.424804 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:58.424804 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:58.424804 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:58.424804 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:58.424804 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:58.424804 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:58.424804 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:58.424804 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:58.424804 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:58.424804 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:58.424804 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:58.424804 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:58.425855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:58.425855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:58.425855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:58.425855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:58.425855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:58.425855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:58.425855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:58.425855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:58.425855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:58.425855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:58.425855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:58.425855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:58.425855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:58.425855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:58.425855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:58.425855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:58.430978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:58.430978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:58.430978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:58.430978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:58.430978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:58.430978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:58.430978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:58.430978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:58.430978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:58.430978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:58.430978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:09:58.430978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:58.430978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 341/670] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 03:09:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:58.430978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d53725-92ed-4350-be9b-96bb6aee406a None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:58.531463 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1d982e-3805-4fb4-ae16-00efb8ca427a None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:58.536489 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d53725-92ed-4350-be9b-96bb6aee406a None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:58.566250 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef611b-007a-4167-8842-b7e4de5acb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:58.567135 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef611b-007a-4167-8842-b7e4de5acb39 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:58.567354 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef611b-007a-4167-8842-b7e4de5acb39 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:58.567635 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef611b-007a-4167-8842-b7e4de5acb39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:58.570276 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe4bc03-bba0-4091-abc4-8c0a3cbf3f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:58.570924 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4bc03-bba0-4091-abc4-8c0a3cbf3f3e None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:58.571198 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4bc03-bba0-4091-abc4-8c0a3cbf3f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:58.571463 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe4bc03-bba0-4091-abc4-8c0a3cbf3f3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:58.572275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe4bc03-bba0-4091-abc4-8c0a3cbf3f3e None admin] RBAC: Adding query filter params (role.id=3451aa51e64642768b2228126d20fa03, scope.project.id=f8398941252346468483ea0c863122f4, user.id=5b3c9c5c8d2e4376b1b70d974d4cb851) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:58.572583 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe4bc03-bba0-4091-abc4-8c0a3cbf3f3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:58.574067 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe4bc03-bba0-4091-abc4-8c0a3cbf3f3e None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:58.578206 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbef611b-007a-4167-8842-b7e4de5acb39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:58.579650 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbef611b-007a-4167-8842-b7e4de5acb39 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:58.580192 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 342/671] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:09:58 2022] GET /identity/v3/role_assignments?user.id=5b3c9c5c8d2e4376b1b70d974d4cb851&scope.project.id=f8398941252346468483ea0c863122f4&role.id=3451aa51e64642768b2228126d20fa03 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:58.580740 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe4bc03-bba0-4091-abc4-8c0a3cbf3f3e None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:58.599485 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-fbef611b-007a-4167-8842-b7e4de5acb39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:58.599485 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:58.599485 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:58.599485 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:58.599485 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:58.599485 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:58.599485 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:58.599485 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:58.599485 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:58.599485 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:58.599485 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:58.599485 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:58.599485 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:09:58.599485 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:09:58.599485 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:09:58.600564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:09:58.600564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:58.600564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:58.600564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:58.600564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:58.600564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:58.600564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:58.600564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:58.600564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:58.600564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:58.600564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:58.600564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:58.600564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:58.600564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:58.600564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:58.600564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:58.600564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:58.601489 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:09:58.601489 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:09:58.601489 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:58.601489 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:58.601489 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 330/672] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:58.601771 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef611b-007a-4167-8842-b7e4de5acb39 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:58.649024 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:58.661409 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:58.700672 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe4bc03-bba0-4091-abc4-8c0a3cbf3f3e None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:58.722869 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef611b-007a-4167-8842-b7e4de5acb39 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:58.738780 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479b261-8c04-42a3-8144-98ca132cabed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:58.739641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479b261-8c04-42a3-8144-98ca132cabed None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:58.739911 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479b261-8c04-42a3-8144-98ca132cabed None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:58.740166 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479b261-8c04-42a3-8144-98ca132cabed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:58.748616 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5479b261-8c04-42a3-8144-98ca132cabed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:58.749923 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5479b261-8c04-42a3-8144-98ca132cabed None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:58.763654 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384ec8c1-f30d-4fe2-8719-dbde901780e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:58.764316 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ec8c1-f30d-4fe2-8719-dbde901780e5 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:58.764590 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ec8c1-f30d-4fe2-8719-dbde901780e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:58.764856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ec8c1-f30d-4fe2-8719-dbde901780e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:58.765165 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-5479b261-8c04-42a3-8144-98ca132cabed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:09:58.765165 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:58.765165 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:58.765165 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:58.765165 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:58.765165 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:58.765165 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:58.765165 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:58.765165 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:58.765165 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:58.765165 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:58.765165 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:58.765165 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:09:58.765165 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:09:58.765165 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:09:58.765995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:09:58.765995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:58.765995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:58.765995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:58.765995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:58.765995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:58.765995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:58.765995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:58.765995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:58.765995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:58.765995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:58.765995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:58.765995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:58.765995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:58.765995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:58.765995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:58.765995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:58.766919 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:09:58.766919 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:09:58.766919 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:09:58.766919 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:09:58.766919 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:09:58.766919 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:09:58.766919 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:09:58.766919 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:58.767350 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384ec8c1-f30d-4fe2-8719-dbde901780e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:58.767350 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384ec8c1-f30d-4fe2-8719-dbde901780e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:58.767473 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 343/673] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:09:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:58.767539 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384ec8c1-f30d-4fe2-8719-dbde901780e5 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:58.768073 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fdcd6b-a46d-4a4c-aa33-033471dbbcbb None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:58.768073 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fdcd6b-a46d-4a4c-aa33-033471dbbcbb None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:58.768242 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fdcd6b-a46d-4a4c-aa33-033471dbbcbb None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:58.771696 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 344/674] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:09:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:09:58.771696 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fdcd6b-a46d-4a4c-aa33-033471dbbcbb None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:58.783547 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 331/675] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:09:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:58.784769 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384ec8c1-f30d-4fe2-8719-dbde901780e5 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:58.886217 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fdcd6b-a46d-4a4c-aa33-033471dbbcbb None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:58.891911 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384ec8c1-f30d-4fe2-8719-dbde901780e5 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:58.901371 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:58.902275 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:58.924367 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a7c5e-02d2-4d9a-880c-871c55504c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:58.925074 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a7c5e-02d2-4d9a-880c-871c55504c74 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:58.925359 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a7c5e-02d2-4d9a-880c-871c55504c74 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:58.925633 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a7c5e-02d2-4d9a-880c-871c55504c74 None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:58.926458 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a7c5e-02d2-4d9a-880c-871c55504c74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:58.926764 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a7c5e-02d2-4d9a-880c-871c55504c74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:58.928172 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a7c5e-02d2-4d9a-880c-871c55504c74 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:58.938823 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca549d31-d2da-4de1-b54b-248c7ea77f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:58.939777 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca549d31-d2da-4de1-b54b-248c7ea77f6f None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:58.940273 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca549d31-d2da-4de1-b54b-248c7ea77f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:58.940700 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 345/676] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:09:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:58.941021 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca549d31-d2da-4de1-b54b-248c7ea77f6f None admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:58.942046 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca549d31-d2da-4de1-b54b-248c7ea77f6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:58.942156 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1dd2e-ac36-41ce-a202-315e4e273d85 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:58.942357 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1dd2e-ac36-41ce-a202-315e4e273d85 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:58.942644 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1dd2e-ac36-41ce-a202-315e4e273d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:58.943119 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca549d31-d2da-4de1-b54b-248c7ea77f6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:58.945214 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca549d31-d2da-4de1-b54b-248c7ea77f6f None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:58.959490 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 332/677] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 03:09:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:58.960510 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-fba1dd2e-ac36-41ce-a202-315e4e273d85 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:09:58.960919 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca549d31-d2da-4de1-b54b-248c7ea77f6f None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:58.961757 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-fba1dd2e-ac36-41ce-a202-315e4e273d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:09:59.035366 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 346/678] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:09:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:09:59.036659 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba1dd2e-ac36-41ce-a202-315e4e273d85 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:59.083920 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca549d31-d2da-4de1-b54b-248c7ea77f6f None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:59.119660 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1944a664-e14d-4f73-8cb6-b4f835f52b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:59.120808 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944a664-e14d-4f73-8cb6-b4f835f52b1c None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:59.121294 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944a664-e14d-4f73-8cb6-b4f835f52b1c None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:59.121742 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944a664-e14d-4f73-8cb6-b4f835f52b1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:59.133150 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1944a664-e14d-4f73-8cb6-b4f835f52b1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:59.134885 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1944a664-e14d-4f73-8cb6-b4f835f52b1c None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:59.150419 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba1dd2e-ac36-41ce-a202-315e4e273d85 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:59.153489 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:59.153663 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-1944a664-e14d-4f73-8cb6-b4f835f52b1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:59.153663 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:59.153663 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:59.153663 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:59.153663 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:59.153663 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:59.153663 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:59.153663 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:59.153663 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:59.153663 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:59.153663 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:59.153663 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:59.153663 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:09:59.153663 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:09:59.153663 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:09:59.154503 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:09:59.154503 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:59.154503 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:59.154503 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:59.154503 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:59.154503 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:59.154503 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:59.154503 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:59.154503 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:09:59.154503 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:59.154503 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:59.154503 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:59.154503 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:59.154503 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:59.154503 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:59.154503 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:59.154503 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:59.155370 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:09:59.155370 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:09:59.155370 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:09:59.155370 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:09:59.162575 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 333/679] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:59.163675 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1944a664-e14d-4f73-8cb6-b4f835f52b1c None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:59.187042 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:59.187949 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484c5de-91cc-4f0f-8700-f54e905f3f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:59.188766 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484c5de-91cc-4f0f-8700-f54e905f3f8d None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:59.189059 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484c5de-91cc-4f0f-8700-f54e905f3f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:59.189326 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1484c5de-91cc-4f0f-8700-f54e905f3f8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:59.196429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1484c5de-91cc-4f0f-8700-f54e905f3f8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:59.197894 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1484c5de-91cc-4f0f-8700-f54e905f3f8d None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:59.211827 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-1484c5de-91cc-4f0f-8700-f54e905f3f8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:09:59.211827 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:59.211827 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:59.211827 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:59.211827 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:59.211827 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:59.211827 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:59.211827 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:59.211827 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:59.211827 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:59.211827 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:59.211827 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:59.211827 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:09:59.211827 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:09:59.211827 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:09:59.212741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:09:59.212741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:59.212741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:59.212741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:59.212741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:59.212741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:59.212741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:59.212741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:59.212741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:59.212741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:59.212741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:59.212741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:59.212741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:59.212741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:59.212741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:59.212741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:59.212741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:59.213658 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:09:59.213658 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:09:59.213658 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:09:59.213658 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:09:59.213658 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:09:59.213658 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:09:59.213658 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:09:59.213658 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:59.213658 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 347/680] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:09:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:59.214616 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484c5de-91cc-4f0f-8700-f54e905f3f8d None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:59.280982 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1944a664-e14d-4f73-8cb6-b4f835f52b1c None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:59.319736 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee5be87-ae8f-440f-bbbc-d13bfb81b1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:59.320396 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee5be87-ae8f-440f-bbbc-d13bfb81b1d2 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:59.320672 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee5be87-ae8f-440f-bbbc-d13bfb81b1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:59.320943 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee5be87-ae8f-440f-bbbc-d13bfb81b1d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:59.321796 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee5be87-ae8f-440f-bbbc-d13bfb81b1d2 None admin] RBAC: Adding query filter params (role.id=3451aa51e64642768b2228126d20fa03, scope.project.id=f8398941252346468483ea0c863122f4, user.id=36dd9b94ceb74eb4a7e9b5bbd5e8d8ba) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:59.322105 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee5be87-ae8f-440f-bbbc-d13bfb81b1d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:59.323532 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee5be87-ae8f-440f-bbbc-d13bfb81b1d2 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:59.330786 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 334/681] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:09:59 2022] GET /identity/v3/role_assignments?user.id=36dd9b94ceb74eb4a7e9b5bbd5e8d8ba&scope.project.id=f8398941252346468483ea0c863122f4&role.id=3451aa51e64642768b2228126d20fa03 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:59.334175 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee5be87-ae8f-440f-bbbc-d13bfb81b1d2 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:59.338362 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1484c5de-91cc-4f0f-8700-f54e905f3f8d None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:59.381807 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f74b5a-4442-4772-a373-3c590b177561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:59.382776 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f74b5a-4442-4772-a373-3c590b177561 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:59.383156 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f74b5a-4442-4772-a373-3c590b177561 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:59.383531 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f74b5a-4442-4772-a373-3c590b177561 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:59.397184 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f74b5a-4442-4772-a373-3c590b177561 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:09:59.406121 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-17f74b5a-4442-4772-a373-3c590b177561 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:09:59.406121 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:59.406121 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:59.406121 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:59.406121 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:59.406121 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:59.406121 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:59.406121 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:59.406121 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:59.406121 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:59.406121 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:59.406121 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:59.406121 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:09:59.406121 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:09:59.406121 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:09:59.406121 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:09:59.407275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:09:59.407275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:09:59.407275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:09:59.407275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:09:59.407275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:59.407275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:59.407275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:59.407275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:59.407275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:59.407275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:59.407275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:59.407275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:59.407275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:59.407275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:59.407275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:59.407275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:59.408323 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:59.408323 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:59.408323 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:59.408323 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:59.408323 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:09:59.408323 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:09:59.408323 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:09:59.408323 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:09:59.408323 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:09:59.408323 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:09:59.408323 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:09:59.408323 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:59.416041 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 348/682] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 03:09:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:59.416041 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f74b5a-4442-4772-a373-3c590b177561 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:59.447420 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:59.471754 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:59.497554 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee5be87-ae8f-440f-bbbc-d13bfb81b1d2 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:59.537164 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f74b5a-4442-4772-a373-3c590b177561 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:59.543950 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a71143f-bebd-4cfc-82ea-413830977cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:59.544626 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71143f-bebd-4cfc-82ea-413830977cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:59.544906 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71143f-bebd-4cfc-82ea-413830977cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:59.545220 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71143f-bebd-4cfc-82ea-413830977cb4 None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:59.545942 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a71143f-bebd-4cfc-82ea-413830977cb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:59.546704 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a71143f-bebd-4cfc-82ea-413830977cb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:59.548089 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a71143f-bebd-4cfc-82ea-413830977cb4 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:59.563312 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 335/683] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:09:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:59.564301 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a71143f-bebd-4cfc-82ea-413830977cb4 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:59.577845 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cf17b2-bccf-4ec4-b8ef-409c52c8faa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:59.578839 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf17b2-bccf-4ec4-b8ef-409c52c8faa4 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:59.579297 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf17b2-bccf-4ec4-b8ef-409c52c8faa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:59.579781 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf17b2-bccf-4ec4-b8ef-409c52c8faa4 None admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:59.580577 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cf17b2-bccf-4ec4-b8ef-409c52c8faa4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:59.580887 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cf17b2-bccf-4ec4-b8ef-409c52c8faa4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:59.582936 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cf17b2-bccf-4ec4-b8ef-409c52c8faa4 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:59.598113 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 349/684] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 03:09:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:59.598963 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cf17b2-bccf-4ec4-b8ef-409c52c8faa4 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:59.683690 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a71143f-bebd-4cfc-82ea-413830977cb4 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:59.706088 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:59.725557 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cf17b2-bccf-4ec4-b8ef-409c52c8faa4 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:59.730597 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02631aa-1b26-45d3-9727-ed0be040151f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:59.731633 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02631aa-1b26-45d3-9727-ed0be040151f None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:59.732120 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02631aa-1b26-45d3-9727-ed0be040151f None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:59.732588 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02631aa-1b26-45d3-9727-ed0be040151f None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:59.733602 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02631aa-1b26-45d3-9727-ed0be040151f None admin] RBAC: Adding query filter params (name=service) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:59.734607 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02631aa-1b26-45d3-9727-ed0be040151f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:59.736722 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02631aa-1b26-45d3-9727-ed0be040151f None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:59.750076 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 336/685] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 03:09:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:59.751409 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02631aa-1b26-45d3-9727-ed0be040151f None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:59.766455 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:09:59.771391 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec7553-b05a-4e4b-9ad0-dd4c7f8ca3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:59.772395 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec7553-b05a-4e4b-9ad0-dd4c7f8ca3f4 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:59.772776 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec7553-b05a-4e4b-9ad0-dd4c7f8ca3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:59.773316 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec7553-b05a-4e4b-9ad0-dd4c7f8ca3f4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:59.781601 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec7553-b05a-4e4b-9ad0-dd4c7f8ca3f4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:59.783100 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec7553-b05a-4e4b-9ad0-dd4c7f8ca3f4 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:59.799250 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-53ec7553-b05a-4e4b-9ad0-dd4c7f8ca3f4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 03:09:59.799250 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:09:59.799250 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:09:59.799250 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:09:59.799250 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:09:59.799250 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:09:59.799250 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:09:59.799250 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:09:59.799250 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:09:59.799250 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:09:59.799250 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:09:59.799250 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:09:59.799250 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:09:59.799250 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:09:59.799250 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:09:59.799250 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:09:59.800442 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:09:59.800442 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:09:59.800442 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:09:59.800442 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:09:59.800442 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:09:59.800442 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:09:59.800442 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:09:59.800442 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:09:59.800442 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:09:59.800442 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:09:59.800442 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:09:59.800442 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:09:59.800442 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:09:59.800442 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:09:59.800442 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:09:59.800442 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:09:59.800442 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:09:59.800442 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:09:59.801641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:09:59.801641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:09:59.801641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:09:59.801641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:09:59.801641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:09:59.801641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:09:59.801641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:09:59.801641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:09:59.801641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 03:09:59.801641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:09:59.808036 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 350/686] 10.222.0.50 () {56 vars in 1093 bytes} [Fri May 27 03:09:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:09:59.809296 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec7553-b05a-4e4b-9ad0-dd4c7f8ca3f4 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:59.891866 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02631aa-1b26-45d3-9727-ed0be040151f None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:59.921795 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec7553-b05a-4e4b-9ad0-dd4c7f8ca3f4 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:09:59.941350 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501859fa-ebeb-45b1-89b5-5678692cc234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:59.941970 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501859fa-ebeb-45b1-89b5-5678692cc234 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:59.942245 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501859fa-ebeb-45b1-89b5-5678692cc234 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:59.942517 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501859fa-ebeb-45b1-89b5-5678692cc234 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:09:59.943346 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501859fa-ebeb-45b1-89b5-5678692cc234 None admin] RBAC: Adding query filter params (role.id=3451aa51e64642768b2228126d20fa03, scope.project.id=f8398941252346468483ea0c863122f4, user.id=4ee30c0d6d694ee8945800157a9fc5b7) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:09:59.943651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501859fa-ebeb-45b1-89b5-5678692cc234 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:09:59.945647 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501859fa-ebeb-45b1-89b5-5678692cc234 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:09:59.954523 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 337/687] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:09:59 2022] GET /identity/v3/role_assignments?user.id=4ee30c0d6d694ee8945800157a9fc5b7&scope.project.id=f8398941252346468483ea0c863122f4&role.id=3451aa51e64642768b2228126d20fa03 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:09:59.955739 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501859fa-ebeb-45b1-89b5-5678692cc234 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:09:59.972416 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d960cf6e-2101-4343-8a2e-f35929ad8344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:09:59.973598 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960cf6e-2101-4343-8a2e-f35929ad8344 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:09:59.974065 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960cf6e-2101-4343-8a2e-f35929ad8344 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:09:59.974440 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960cf6e-2101-4343-8a2e-f35929ad8344 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:00.003333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d960cf6e-2101-4343-8a2e-f35929ad8344 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:00.005342 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d960cf6e-2101-4343-8a2e-f35929ad8344 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:00.012625 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:00.037312 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:00.045954 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-d960cf6e-2101-4343-8a2e-f35929ad8344 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:00.045954 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:00.045954 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:00.045954 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:00.045954 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:00.045954 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:00.045954 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:00.045954 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:00.045954 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:00.045954 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:00.045954 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:00.045954 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:00.045954 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:00.045954 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:00.045954 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:00.046914 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:00.046914 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:00.046914 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:00.046914 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:00.046914 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:00.046914 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:00.046914 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:00.046914 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:00.046914 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:00.046914 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:00.046914 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:00.046914 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:00.046914 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:00.046914 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:00.046914 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:00.046914 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:00.046914 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:00.047789 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:00.047789 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:00.047789 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:00.047789 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:00.057154 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 351/688] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:09:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:00.058103 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d960cf6e-2101-4343-8a2e-f35929ad8344 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:00.091066 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501859fa-ebeb-45b1-89b5-5678692cc234 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:00.181363 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d960cf6e-2101-4343-8a2e-f35929ad8344 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:00.187122 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeccca2-a50c-470f-a425-b6a9605a0a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:00.187831 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeccca2-a50c-470f-a425-b6a9605a0a1b None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:00.188109 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeccca2-a50c-470f-a425-b6a9605a0a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:00.188384 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeccca2-a50c-470f-a425-b6a9605a0a1b None admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:00.189300 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeccca2-a50c-470f-a425-b6a9605a0a1b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:00.189612 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeccca2-a50c-470f-a425-b6a9605a0a1b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:00.191145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeccca2-a50c-470f-a425-b6a9605a0a1b None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:00.220313 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f30031-2dc8-4afa-b2b3-82898faaeb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:00.220907 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f30031-2dc8-4afa-b2b3-82898faaeb64 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:00.221231 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f30031-2dc8-4afa-b2b3-82898faaeb64 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:00.221504 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f30031-2dc8-4afa-b2b3-82898faaeb64 None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:00.222229 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f30031-2dc8-4afa-b2b3-82898faaeb64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:00.222528 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f30031-2dc8-4afa-b2b3-82898faaeb64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:00.223965 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f30031-2dc8-4afa-b2b3-82898faaeb64 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:00.231134 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 338/689] 10.222.0.50 () {56 vars in 1133 bytes} [Fri May 27 03:09:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:00.236883 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeccca2-a50c-470f-a425-b6a9605a0a1b None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:00.238147 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 352/690] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:10:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:00.244990 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f30031-2dc8-4afa-b2b3-82898faaeb64 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:00.352353 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:00.370433 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeccca2-a50c-470f-a425-b6a9605a0a1b None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:00.404783 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f30031-2dc8-4afa-b2b3-82898faaeb64 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:00.413356 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:00.425600 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529ea9b9-1abf-4e0d-aeb9-145eee9f8a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:00.427127 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ea9b9-1abf-4e0d-aeb9-145eee9f8a32 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:00.427611 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ea9b9-1abf-4e0d-aeb9-145eee9f8a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:00.428048 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529ea9b9-1abf-4e0d-aeb9-145eee9f8a32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:00.449222 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bbce38-2039-4961-b714-00e1121a817f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:00.450848 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bbce38-2039-4961-b714-00e1121a817f None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:00.451640 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bbce38-2039-4961-b714-00e1121a817f None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:00.452116 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bbce38-2039-4961-b714-00e1121a817f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:00.463510 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bbce38-2039-4961-b714-00e1121a817f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:00.465695 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bbce38-2039-4961-b714-00e1121a817f None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:00.485017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-84bbce38-2039-4961-b714-00e1121a817f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 03:10:00.485017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:00.485017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:00.485017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:00.485017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:00.485017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:00.485017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:00.485017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:00.485017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:00.485017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:00.485017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:00.485017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:00.485017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:10:00.485017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:10:00.485017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:10:00.485017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:10:00.485982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:00.485982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:00.485982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:10:00.485982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:00.485982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:10:00.485982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:00.485982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:00.485982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:00.485982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:00.485982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:00.485982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:00.485982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:00.485982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:00.485982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:00.485982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:00.485982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:00.485982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:00.485982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:00.486948 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:00.486948 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:00.486948 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:10:00.486948 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:10:00.486948 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:10:00.486948 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:10:00.486948 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:10:00.486948 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:10:00.486948 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 03:10:00.486948 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:00.490426 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 353/691] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 03:10:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:00.496198 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bbce38-2039-4961-b714-00e1121a817f None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:00.508259 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529ea9b9-1abf-4e0d-aeb9-145eee9f8a32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:00.509812 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529ea9b9-1abf-4e0d-aeb9-145eee9f8a32 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:00.612317 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 339/692] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:10:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 03:10:00.625063 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529ea9b9-1abf-4e0d-aeb9-145eee9f8a32 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:00.673378 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:00.680304 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bbce38-2039-4961-b714-00e1121a817f None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:00.717471 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:00.722970 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c72f8-91a3-41ac-9cac-55ea249700e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:00.723905 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c72f8-91a3-41ac-9cac-55ea249700e0 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:00.724087 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c72f8-91a3-41ac-9cac-55ea249700e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:00.724368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7c72f8-91a3-41ac-9cac-55ea249700e0 None admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:00.725278 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7c72f8-91a3-41ac-9cac-55ea249700e0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:00.725593 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7c72f8-91a3-41ac-9cac-55ea249700e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:00.727400 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7c72f8-91a3-41ac-9cac-55ea249700e0 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:00.757873 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 354/693] 10.222.0.50 () {56 vars in 1135 bytes} [Fri May 27 03:10:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:00.770210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c72f8-91a3-41ac-9cac-55ea249700e0 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:00.779091 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529ea9b9-1abf-4e0d-aeb9-145eee9f8a32 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:00.845684 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c59a96-2618-462e-8083-d9bf3967e5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:00.846508 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c59a96-2618-462e-8083-d9bf3967e5fb None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:00.846782 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c59a96-2618-462e-8083-d9bf3967e5fb None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:00.847046 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c59a96-2618-462e-8083-d9bf3967e5fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:00.870039 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c59a96-2618-462e-8083-d9bf3967e5fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:00.871531 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c59a96-2618-462e-8083-d9bf3967e5fb None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:00.886659 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7c72f8-91a3-41ac-9cac-55ea249700e0 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:00.897318 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-70c59a96-2618-462e-8083-d9bf3967e5fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:00.897318 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:00.897318 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:00.897318 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:00.897318 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:00.897318 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:00.897318 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:00.897318 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:00.897318 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:00.897318 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:00.897318 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:00.897318 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:00.897318 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:00.897318 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:00.897318 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:00.898276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:00.898276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:00.898276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:00.898276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:00.898276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:00.898276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:00.898276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:00.898276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:00.898276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:00.898276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:00.898276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:00.898276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:00.898276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:00.898276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:00.898276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:00.898276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:00.898276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:00.899142 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:00.899142 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:00.899142 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:00.899142 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:00.901528 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 340/694] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:00.906950 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c59a96-2618-462e-8083-d9bf3967e5fb None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:00.949260 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0976ee3e-030f-4227-a43b-984c22f6caff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:00.950357 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0976ee3e-030f-4227-a43b-984c22f6caff None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:00.950866 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0976ee3e-030f-4227-a43b-984c22f6caff None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:00.951337 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0976ee3e-030f-4227-a43b-984c22f6caff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:01.032147 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:01.058188 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:01.062553 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0976ee3e-030f-4227-a43b-984c22f6caff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:01.064281 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0976ee3e-030f-4227-a43b-984c22f6caff None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:01.083033 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c59a96-2618-462e-8083-d9bf3967e5fb None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:01.196396 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 355/695] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:10:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 03:10:01.198315 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206749ce-1983-4b02-a7f9-fe9e13cf4425 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:01.198810 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206749ce-1983-4b02-a7f9-fe9e13cf4425 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:01.199334 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206749ce-1983-4b02-a7f9-fe9e13cf4425 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:01.201235 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 356/696] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:01.202213 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206749ce-1983-4b02-a7f9-fe9e13cf4425 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:01.298474 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:01.328182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206749ce-1983-4b02-a7f9-fe9e13cf4425 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:01.341730 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eb5a3c-1ae4-4fd8-ae7d-b95fb891a016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:01.342327 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb5a3c-1ae4-4fd8-ae7d-b95fb891a016 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:01.342603 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb5a3c-1ae4-4fd8-ae7d-b95fb891a016 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:01.342879 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb5a3c-1ae4-4fd8-ae7d-b95fb891a016 None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:01.343617 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eb5a3c-1ae4-4fd8-ae7d-b95fb891a016 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:01.343920 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eb5a3c-1ae4-4fd8-ae7d-b95fb891a016 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:01.349567 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eb5a3c-1ae4-4fd8-ae7d-b95fb891a016 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:01.370150 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 341/697] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:10:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 463 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:01.371192 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e2c49-fc3e-46f5-afe2-8e9821b5b2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:01.371439 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493cd363-72cc-47c5-8099-41e2d6f61c88 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:01.371723 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493cd363-72cc-47c5-8099-41e2d6f61c88 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:01.371995 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493cd363-72cc-47c5-8099-41e2d6f61c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:01.372905 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e2c49-fc3e-46f5-afe2-8e9821b5b2e1 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:01.373418 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e2c49-fc3e-46f5-afe2-8e9821b5b2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:01.373885 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e2c49-fc3e-46f5-afe2-8e9821b5b2e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:01.385521 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0e2c49-fc3e-46f5-afe2-8e9821b5b2e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:01.387330 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0e2c49-fc3e-46f5-afe2-8e9821b5b2e1 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:01.393433 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-493cd363-72cc-47c5-8099-41e2d6f61c88 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:01.396838 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-493cd363-72cc-47c5-8099-41e2d6f61c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:01.410641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-3a0e2c49-fc3e-46f5-afe2-8e9821b5b2e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:01.410641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:01.410641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:01.410641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:01.410641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:01.410641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:01.410641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:01.410641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:01.410641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:01.410641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:01.410641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:01.410641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:01.410641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:01.410641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:01.410641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:01.411775 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:01.411775 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:01.411775 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:01.411775 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:01.411775 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:01.411775 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:01.411775 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:01.411775 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:01.411775 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:01.411775 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:01.411775 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:01.411775 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:01.411775 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:01.411775 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:01.411775 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:01.411775 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:01.411775 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:01.412890 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:01.412890 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:01.412890 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:01.412890 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:01.419179 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 357/698] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:01.420221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e2c49-fc3e-46f5-afe2-8e9821b5b2e1 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:01.479381 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 342/699] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:01.480376 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493cd363-72cc-47c5-8099-41e2d6f61c88 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:01.558926 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:01.614075 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493cd363-72cc-47c5-8099-41e2d6f61c88 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:01.642603 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e2c49-fc3e-46f5-afe2-8e9821b5b2e1 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:01.671352 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f815fae-4cbc-4fe3-b1f2-ad08c22fd994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:01.671763 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f815fae-4cbc-4fe3-b1f2-ad08c22fd994 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:01.672045 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f815fae-4cbc-4fe3-b1f2-ad08c22fd994 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:01.672315 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f815fae-4cbc-4fe3-b1f2-ad08c22fd994 None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:01.673091 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f815fae-4cbc-4fe3-b1f2-ad08c22fd994 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:01.673387 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f815fae-4cbc-4fe3-b1f2-ad08c22fd994 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:01.674752 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f815fae-4cbc-4fe3-b1f2-ad08c22fd994 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:01.679656 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:01.690352 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 343/700] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:10:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:01.690822 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4921efc4-bd33-400e-bd6c-5ac04c71f6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:01.691250 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f815fae-4cbc-4fe3-b1f2-ad08c22fd994 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:01.692485 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4921efc4-bd33-400e-bd6c-5ac04c71f6a4 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:01.693040 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4921efc4-bd33-400e-bd6c-5ac04c71f6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:01.693509 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4921efc4-bd33-400e-bd6c-5ac04c71f6a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:01.703400 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4921efc4-bd33-400e-bd6c-5ac04c71f6a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:01.705820 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4921efc4-bd33-400e-bd6c-5ac04c71f6a4 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:01.726030 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-4921efc4-bd33-400e-bd6c-5ac04c71f6a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:01.726030 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:01.726030 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:01.726030 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:01.726030 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:01.726030 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:01.726030 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:01.726030 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:01.726030 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:01.726030 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:01.726030 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:01.726030 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:01.726030 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:10:01.726030 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:10:01.726030 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:10:01.726972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:10:01.726972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:01.726972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:01.726972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:01.726972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:01.726972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:01.726972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:01.726972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:01.726972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:01.726972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:01.726972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:01.726972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:01.726972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:01.726972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:01.726972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:01.726972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:01.726972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:01.727833 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:10:01.727833 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:10:01.727833 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:10:01.727833 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:10:01.727833 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:10:01.727833 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:10:01.727833 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:01.727833 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:01.736022 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 358/701] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:10:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:01.737206 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4921efc4-bd33-400e-bd6c-5ac04c71f6a4 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:01.801104 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f815fae-4cbc-4fe3-b1f2-ad08c22fd994 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:01.830420 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:01.837055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe62c03d-7032-453f-83cc-d242485e11d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:01.837761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62c03d-7032-453f-83cc-d242485e11d2 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:01.838038 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62c03d-7032-453f-83cc-d242485e11d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:01.838305 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62c03d-7032-453f-83cc-d242485e11d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:01.848966 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe62c03d-7032-453f-83cc-d242485e11d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:01.850849 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe62c03d-7032-453f-83cc-d242485e11d2 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:01.861555 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4921efc4-bd33-400e-bd6c-5ac04c71f6a4 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:01.870136 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-fe62c03d-7032-453f-83cc-d242485e11d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:01.870136 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:01.870136 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:01.870136 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:01.870136 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:01.870136 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:01.870136 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:01.870136 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:01.870136 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:01.870136 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:01.870136 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:01.870136 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:01.870136 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:01.870136 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:01.870136 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:01.871017 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:01.871017 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:01.871017 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:01.871017 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:01.871017 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:01.871017 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:01.871017 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:01.871017 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:01.871017 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:01.871017 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:01.871017 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:01.871017 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:01.871017 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:01.871017 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:01.871017 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:01.871017 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:01.871017 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:01.871891 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:01.871891 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:01.871891 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:01.871891 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:01.872702 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 344/702] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:01.873645 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe62c03d-7032-453f-83cc-d242485e11d2 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:01.899780 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b87d3f-36ed-4806-9c46-a786bc929b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:01.900573 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b87d3f-36ed-4806-9c46-a786bc929b8b None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:01.900847 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b87d3f-36ed-4806-9c46-a786bc929b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:01.901159 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b87d3f-36ed-4806-9c46-a786bc929b8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:01.907824 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b87d3f-36ed-4806-9c46-a786bc929b8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:01.909914 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b87d3f-36ed-4806-9c46-a786bc929b8b None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:01.924003 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-c7b87d3f-36ed-4806-9c46-a786bc929b8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:01.924003 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:01.924003 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:01.924003 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:01.924003 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:01.924003 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:01.924003 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:01.924003 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:01.924003 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:01.924003 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:01.924003 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:01.924003 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:01.924003 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:10:01.924003 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:10:01.924003 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:10:01.924903 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:10:01.924903 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:01.924903 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:01.924903 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:01.924903 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:01.924903 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:01.924903 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:01.924903 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:01.924903 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:01.924903 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:01.924903 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:01.924903 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:01.924903 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:01.924903 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:01.924903 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:01.924903 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:01.924903 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:01.925959 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:10:01.925959 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:10:01.925959 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:10:01.925959 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:10:01.925959 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:10:01.925959 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:10:01.925959 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:01.925959 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:01.925959 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 359/703] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:10:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:01.927153 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003b76d-bfb2-4e1e-adc5-54edd7782c9c None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:01.927153 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003b76d-bfb2-4e1e-adc5-54edd7782c9c None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:01.927320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003b76d-bfb2-4e1e-adc5-54edd7782c9c None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:01.928669 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 360/704] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:01.933755 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003b76d-bfb2-4e1e-adc5-54edd7782c9c None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:01.980936 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe62c03d-7032-453f-83cc-d242485e11d2 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:01.985761 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:02.014518 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f13384-960f-452d-87eb-5a9e625b097c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:02.015183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f13384-960f-452d-87eb-5a9e625b097c None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:02.015455 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f13384-960f-452d-87eb-5a9e625b097c None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:02.015728 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f13384-960f-452d-87eb-5a9e625b097c None admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:02.016508 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f13384-960f-452d-87eb-5a9e625b097c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:02.016812 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f13384-960f-452d-87eb-5a9e625b097c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:02.018270 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f13384-960f-452d-87eb-5a9e625b097c None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:02.031003 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 345/705] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 03:10:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:02.034477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f13384-960f-452d-87eb-5a9e625b097c None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:02.042592 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003b76d-bfb2-4e1e-adc5-54edd7782c9c None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:02.079557 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bab98d-15f8-4715-a388-517c4fb9735f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:02.080224 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bab98d-15f8-4715-a388-517c4fb9735f None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:02.080604 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bab98d-15f8-4715-a388-517c4fb9735f None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:02.080820 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bab98d-15f8-4715-a388-517c4fb9735f None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:02.081574 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bab98d-15f8-4715-a388-517c4fb9735f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:02.081871 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bab98d-15f8-4715-a388-517c4fb9735f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:02.082828 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:02.083422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bab98d-15f8-4715-a388-517c4fb9735f None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:02.097059 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 361/706] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:10:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:02.097059 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a252cc-3b19-4345-b645-41be67d7f2f1 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:02.097355 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a252cc-3b19-4345-b645-41be67d7f2f1 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:02.097576 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a252cc-3b19-4345-b645-41be67d7f2f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:02.118705 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-23a252cc-3b19-4345-b645-41be67d7f2f1 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:02.119881 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-23a252cc-3b19-4345-b645-41be67d7f2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:02.154361 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f13384-960f-452d-87eb-5a9e625b097c None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:02.188506 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 362/707] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:02.188933 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94791e3f-6c88-426b-b790-bac287bcb34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:02.189617 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94791e3f-6c88-426b-b790-bac287bcb34f None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:02.189925 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a252cc-3b19-4345-b645-41be67d7f2f1 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:02.190039 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94791e3f-6c88-426b-b790-bac287bcb34f None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:02.190220 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94791e3f-6c88-426b-b790-bac287bcb34f None admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:02.191005 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94791e3f-6c88-426b-b790-bac287bcb34f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:02.191325 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94791e3f-6c88-426b-b790-bac287bcb34f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:02.193054 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94791e3f-6c88-426b-b790-bac287bcb34f None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:02.208694 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 346/708] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 03:10:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:02.209601 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94791e3f-6c88-426b-b790-bac287bcb34f None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:02.239148 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:02.310588 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a252cc-3b19-4345-b645-41be67d7f2f1 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:02.321727 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94791e3f-6c88-426b-b790-bac287bcb34f None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:02.323256 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:02.348103 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3922e7-349d-4a85-b687-8272d1cbbb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:02.348807 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3922e7-349d-4a85-b687-8272d1cbbb6c None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:02.349201 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3922e7-349d-4a85-b687-8272d1cbbb6c None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:02.349597 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3922e7-349d-4a85-b687-8272d1cbbb6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:02.350500 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3922e7-349d-4a85-b687-8272d1cbbb6c None admin] RBAC: Adding query filter params (role.id=b4c524dd7bb44e94bd3b42d86e1cdf42, scope.project.id=f8398941252346468483ea0c863122f4, user.id=1a699988f08a4001b3b7d764f01f4fc5) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:02.350901 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3922e7-349d-4a85-b687-8272d1cbbb6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:02.352552 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3922e7-349d-4a85-b687-8272d1cbbb6c None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:02.357178 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15f4ee-131a-4389-8f51-fa8b88e4e92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:02.357926 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15f4ee-131a-4389-8f51-fa8b88e4e92a None admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:02.358268 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15f4ee-131a-4389-8f51-fa8b88e4e92a None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:02.369400 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15f4ee-131a-4389-8f51-fa8b88e4e92a None admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:02.369614 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 363/709] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:10:02 2022] GET /identity/v3/role_assignments?user.id=1a699988f08a4001b3b7d764f01f4fc5&scope.project.id=f8398941252346468483ea0c863122f4&role.id=b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:02.369735 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15f4ee-131a-4389-8f51-fa8b88e4e92a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:02.369826 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3922e7-349d-4a85-b687-8272d1cbbb6c None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:02.369912 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15f4ee-131a-4389-8f51-fa8b88e4e92a None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:02.401439 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 347/710] 10.222.0.50 () {60 vars in 1132 bytes} [Fri May 27 03:10:02 2022] POST /identity/v3/users => generated 254 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:10:02.402494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15f4ee-131a-4389-8f51-fa8b88e4e92a None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:02.576504 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15f4ee-131a-4389-8f51-fa8b88e4e92a None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:02.583477 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:02.586328 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3922e7-349d-4a85-b687-8272d1cbbb6c None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:02.646295 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a3bee9-5aed-4252-a4ee-8d12726700e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:02.646927 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a3bee9-5aed-4252-a4ee-8d12726700e6 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:02.647159 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a3bee9-5aed-4252-a4ee-8d12726700e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:02.647429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a3bee9-5aed-4252-a4ee-8d12726700e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:02.648279 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a3bee9-5aed-4252-a4ee-8d12726700e6 None admin] RBAC: Adding query filter params (role.id=3451aa51e64642768b2228126d20fa03, scope.project.id=f8398941252346468483ea0c863122f4, user.id=3de53e2f0fba4e0fac70887067c1963d) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:02.648588 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a3bee9-5aed-4252-a4ee-8d12726700e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:02.650115 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a3bee9-5aed-4252-a4ee-8d12726700e6 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:02.652913 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f216ef49-7281-4e27-bf0a-6342739624cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:02.653783 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f216ef49-7281-4e27-bf0a-6342739624cb None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:02.654122 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f216ef49-7281-4e27-bf0a-6342739624cb None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:02.654394 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f216ef49-7281-4e27-bf0a-6342739624cb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:02.655107 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f216ef49-7281-4e27-bf0a-6342739624cb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:02.656573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f216ef49-7281-4e27-bf0a-6342739624cb None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:02.665066 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 364/711] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:10:02 2022] GET /identity/v3/role_assignments?user.id=3de53e2f0fba4e0fac70887067c1963d&scope.project.id=f8398941252346468483ea0c863122f4&role.id=3451aa51e64642768b2228126d20fa03 => generated 608 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:02.683494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-f216ef49-7281-4e27-bf0a-6342739624cb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 03:10:02.683494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:02.683494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:10:02.683494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:10:02.683494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:10:02.683494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:10:02.683494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:10:02.683494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:02.683494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 03:10:02.683494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:02.683494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:10:02.683494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:02.683494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:02.683494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:02.683494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:02.683494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:02.685103 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:02.685103 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:02.685103 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:02.685103 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:02.685103 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:02.685103 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:02.685103 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:02.685103 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:10:02.685103 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:10:02.685103 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:10:02.685103 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:10:02.685103 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:02.685103 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:02.685103 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:02.685103 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:02.685103 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:02.686262 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:02.686262 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:02.686262 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:02.686262 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:02.686262 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:02.686262 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:02.686262 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:02.686262 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:02.686262 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:02.686262 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:02.686262 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:02.686262 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:10:02.686262 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:02.686262 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 03:10:02.686262 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:02.686262 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 348/712] 10.222.0.50 () {56 vars in 1129 bytes} [Fri May 27 03:10:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:02.693930 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a3bee9-5aed-4252-a4ee-8d12726700e6 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:02.767254 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2da06d-7a1d-48d8-a34e-e53fdfba4213 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:02.767543 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2da06d-7a1d-48d8-a34e-e53fdfba4213 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:02.767823 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2da06d-7a1d-48d8-a34e-e53fdfba4213 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:02.769465 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 349/713] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:02.776271 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da43372-8beb-41fc-851a-a31e86064ac6 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:02.776552 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da43372-8beb-41fc-851a-a31e86064ac6 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:02.776827 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da43372-8beb-41fc-851a-a31e86064ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:02.798917 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-4da43372-8beb-41fc-851a-a31e86064ac6 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:02.800602 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-4da43372-8beb-41fc-851a-a31e86064ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:02.812847 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a3bee9-5aed-4252-a4ee-8d12726700e6 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:02.849343 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:02.852911 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822e0580-a3f2-428f-a906-6ac2f17067cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:02.853653 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e0580-a3f2-428f-a906-6ac2f17067cc None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:02.853939 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e0580-a3f2-428f-a906-6ac2f17067cc None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:02.854386 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e0580-a3f2-428f-a906-6ac2f17067cc None admin] PATH_INFO: `/v3/services` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:02.854971 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822e0580-a3f2-428f-a906-6ac2f17067cc None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:02.856338 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822e0580-a3f2-428f-a906-6ac2f17067cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:02.858773 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822e0580-a3f2-428f-a906-6ac2f17067cc None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:02.865263 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 365/714] 10.222.0.50 () {56 vars in 1124 bytes} [Fri May 27 03:10:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:02.870649 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822e0580-a3f2-428f-a906-6ac2f17067cc None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:02.902906 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 350/715] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:02.923354 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:02.931695 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da43372-8beb-41fc-851a-a31e86064ac6 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:03.009421 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822e0580-a3f2-428f-a906-6ac2f17067cc None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:03.058179 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f16dfc7-c337-46a2-8f7e-c901f73359a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:03.059307 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16dfc7-c337-46a2-8f7e-c901f73359a7 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:03.059656 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16dfc7-c337-46a2-8f7e-c901f73359a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:03.059939 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16dfc7-c337-46a2-8f7e-c901f73359a7 None admin] PATH_INFO: `/v3/services` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:03.060773 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f16dfc7-c337-46a2-8f7e-c901f73359a7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:03.061206 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f16dfc7-c337-46a2-8f7e-c901f73359a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:03.062640 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f16dfc7-c337-46a2-8f7e-c901f73359a7 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:03.064003 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da43372-8beb-41fc-851a-a31e86064ac6 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:03.070088 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 366/716] 10.222.0.50 () {56 vars in 1124 bytes} [Fri May 27 03:10:02 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:03.103862 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed59bc9-d990-4ef1-8d88-29a38e67c831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:03.104690 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed59bc9-d990-4ef1-8d88-29a38e67c831 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:03.105002 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed59bc9-d990-4ef1-8d88-29a38e67c831 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:03.105271 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed59bc9-d990-4ef1-8d88-29a38e67c831 None admin] PATH_INFO: `/v3/services/network` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:03.106069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed59bc9-d990-4ef1-8d88-29a38e67c831 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:03.107498 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed59bc9-d990-4ef1-8d88-29a38e67c831 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:03.120609 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-fed59bc9-d990-4ef1-8d88-29a38e67c831 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 27 03:10:03.120609 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:03.120609 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:10:03.120609 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:10:03.120609 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:10:03.120609 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:10:03.120609 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:10:03.120609 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:03.120609 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 03:10:03.120609 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:03.120609 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:10:03.120609 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:03.120609 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:03.120609 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:03.120609 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:03.120609 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:03.120609 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:03.121944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:03.121944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:03.121944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:03.121944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:03.121944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:03.121944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:03.121944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:10:03.121944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:10:03.121944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:10:03.121944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:10:03.121944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:03.121944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:03.121944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:03.121944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:03.121944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:03.121944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:03.122760 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:03.122760 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:03.122760 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:03.122760 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:03.122760 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:03.122760 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:03.122760 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:03.122760 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:03.122760 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:03.122760 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:03.122760 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:10:03.122760 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:03.122760 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 03:10:03.122760 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:03.127225 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 351/717] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:10:02 2022] GET /identity/v3/services/network => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:03.129043 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f16dfc7-c337-46a2-8f7e-c901f73359a7 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:03.233154 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:03.242977 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f16dfc7-c337-46a2-8f7e-c901f73359a7 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:03.281707 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c94e7ab-d358-4681-ba28-be3771ad1c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:03.282387 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94e7ab-d358-4681-ba28-be3771ad1c47 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:03.282661 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94e7ab-d358-4681-ba28-be3771ad1c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:03.282931 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94e7ab-d358-4681-ba28-be3771ad1c47 None admin] PATH_INFO: `/v3/services` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:03.283646 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c94e7ab-d358-4681-ba28-be3771ad1c47 None admin] RBAC: Adding query filter params (name=network) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:03.283945 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c94e7ab-d358-4681-ba28-be3771ad1c47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:03.286176 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c94e7ab-d358-4681-ba28-be3771ad1c47 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:03.291090 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 367/718] 10.222.0.50 () {56 vars in 1112 bytes} [Fri May 27 03:10:03 2022] GET /identity/v3/services?name=network => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:03.297257 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed59bc9-d990-4ef1-8d88-29a38e67c831 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:03.318619 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:03.401878 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed59bc9-d990-4ef1-8d88-29a38e67c831 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:03.437156 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39c981e-3e56-4144-8f79-a5ca7d69dc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:03.437839 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c981e-3e56-4144-8f79-a5ca7d69dc48 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:03.438114 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c981e-3e56-4144-8f79-a5ca7d69dc48 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:03.438382 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39c981e-3e56-4144-8f79-a5ca7d69dc48 None admin] PATH_INFO: `/v3/services` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:03.439093 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c981e-3e56-4144-8f79-a5ca7d69dc48 None admin] RBAC: Adding query filter params (type=network) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:03.439392 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c981e-3e56-4144-8f79-a5ca7d69dc48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:03.440788 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39c981e-3e56-4144-8f79-a5ca7d69dc48 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:03.448229 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 352/719] 10.222.0.50 () {56 vars in 1112 bytes} [Fri May 27 03:10:03 2022] GET /identity/v3/services?type=network => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:04.959757 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af927e99-85e7-43f3-a8e1-80bb809d058b None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:04.960286 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af927e99-85e7-43f3-a8e1-80bb809d058b None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:04.960286 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af927e99-85e7-43f3-a8e1-80bb809d058b None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:04.962092 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 368/720] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:04.967619 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8c077-24be-43f9-82ee-d5f1a93cbac2 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:04.968069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8c077-24be-43f9-82ee-d5f1a93cbac2 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:04.968146 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8c077-24be-43f9-82ee-d5f1a93cbac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:04.986418 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-75e8c077-24be-43f9-82ee-d5f1a93cbac2 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:04.987680 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-75e8c077-24be-43f9-82ee-d5f1a93cbac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:05.088023 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 353/721] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:05.122388 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af927e99-85e7-43f3-a8e1-80bb809d058b None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:05.258319 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af927e99-85e7-43f3-a8e1-80bb809d058b None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:05.322668 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e4ed1-28b1-46f2-96d6-078ff1e88776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:05.323500 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e4ed1-28b1-46f2-96d6-078ff1e88776 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:05.323781 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e4ed1-28b1-46f2-96d6-078ff1e88776 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:05.324050 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e4ed1-28b1-46f2-96d6-078ff1e88776 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:05.329793 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106a211-9066-48b9-9b3d-190d0c16babe None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:05.330341 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106a211-9066-48b9-9b3d-190d0c16babe None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:05.330835 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106a211-9066-48b9-9b3d-190d0c16babe None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:05.332700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 354/722] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:05.337689 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e4ed1-28b1-46f2-96d6-078ff1e88776 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:05.345337 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-cd9e4ed1-28b1-46f2-96d6-078ff1e88776 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:10:05.345337 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:05.345337 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:05.345337 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:05.345337 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:05.345337 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:05.345337 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:05.345337 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:05.345337 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:05.345337 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:05.345337 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:05.345337 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:05.345337 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:05.345337 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:05.345337 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:05.345337 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:05.346263 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:05.346263 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:05.346263 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:05.346263 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:05.346263 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:05.346263 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:05.346263 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:05.346263 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:05.346263 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:05.346263 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:05.346263 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:05.346263 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:05.346263 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:05.346263 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:05.346263 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:05.346263 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:05.347095 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:05.347095 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:05.347095 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:05.347095 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:05.347095 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:05.347095 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:05.347095 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:05.347095 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:05.347095 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:05.347095 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:05.347095 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:10:05.347095 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:05.347095 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 369/723] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 03:10:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:05.352754 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e4ed1-28b1-46f2-96d6-078ff1e88776 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:05.352935 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da3cc8-d56a-483c-b2ff-713255c7e645 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:05.353058 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da3cc8-d56a-483c-b2ff-713255c7e645 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:05.353343 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da3cc8-d56a-483c-b2ff-713255c7e645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:05.386348 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:05.386592 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-b1da3cc8-d56a-483c-b2ff-713255c7e645 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:05.387672 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-b1da3cc8-d56a-483c-b2ff-713255c7e645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:05.394944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:05.514827 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 355/724] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:05.536411 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e4ed1-28b1-46f2-96d6-078ff1e88776 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:05.587354 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da3cc8-d56a-483c-b2ff-713255c7e645 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:05.587580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa722af1-1bd2-4725-b026-15dab4aed82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:05.588125 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa722af1-1bd2-4725-b026-15dab4aed82c None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:05.588409 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa722af1-1bd2-4725-b026-15dab4aed82c None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:05.588688 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa722af1-1bd2-4725-b026-15dab4aed82c None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:05.589524 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa722af1-1bd2-4725-b026-15dab4aed82c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:05.589832 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa722af1-1bd2-4725-b026-15dab4aed82c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:05.594793 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa722af1-1bd2-4725-b026-15dab4aed82c None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:05.602665 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 370/725] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:10:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:05.608421 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa722af1-1bd2-4725-b026-15dab4aed82c None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:05.721334 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da3cc8-d56a-483c-b2ff-713255c7e645 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:05.737661 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:05.759701 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d471b089-1a10-48c1-a560-73d5595209b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:05.760836 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471b089-1a10-48c1-a560-73d5595209b0 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:05.761325 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471b089-1a10-48c1-a560-73d5595209b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:05.761729 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471b089-1a10-48c1-a560-73d5595209b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:05.767591 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa722af1-1bd2-4725-b026-15dab4aed82c None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:05.774540 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:05.776654 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d471b089-1a10-48c1-a560-73d5595209b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:05.790522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-d471b089-1a10-48c1-a560-73d5595209b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:10:05.790522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:05.790522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:05.790522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:05.790522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:05.790522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:05.790522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:05.790522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:05.790522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:05.790522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:05.790522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:05.790522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:05.790522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:05.790522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:05.790522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:05.790522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:05.791682 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:05.791682 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:05.791682 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:05.791682 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:05.791682 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:05.791682 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:05.791682 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:05.791682 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:05.791682 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:05.791682 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:05.791682 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:05.791682 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:05.791682 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:05.791682 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:05.791682 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:05.791682 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:05.792631 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:05.792631 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:05.792631 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:05.792631 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:05.792631 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:05.792631 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:05.792631 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:05.792631 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:05.792631 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:05.792631 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:05.792631 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:10:05.792631 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:05.795135 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 356/726] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 03:10:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:10:05.796611 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3ad96-74e4-42cc-adec-7641004992e8 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:05.797056 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3ad96-74e4-42cc-adec-7641004992e8 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:05.797707 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3ad96-74e4-42cc-adec-7641004992e8 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:05.799575 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 357/727] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:05.800527 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3ad96-74e4-42cc-adec-7641004992e8 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:05.817143 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676afb5-1b52-490a-8bfd-e06c049cf4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:05.817992 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676afb5-1b52-490a-8bfd-e06c049cf4d3 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:05.818276 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676afb5-1b52-490a-8bfd-e06c049cf4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:05.818549 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676afb5-1b52-490a-8bfd-e06c049cf4d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:05.832350 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5676afb5-1b52-490a-8bfd-e06c049cf4d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:05.833883 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5676afb5-1b52-490a-8bfd-e06c049cf4d3 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:05.852208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-5676afb5-1b52-490a-8bfd-e06c049cf4d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:05.852208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:05.852208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:05.852208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:05.852208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:05.852208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:05.852208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:05.852208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:05.852208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:05.852208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:05.852208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:05.852208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:05.852208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:05.852208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:05.852208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:05.853157 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:05.853157 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:05.853157 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:05.853157 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:05.853157 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:05.853157 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:05.853157 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:05.853157 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:05.853157 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:05.853157 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:05.853157 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:05.853157 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:05.853157 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:05.853157 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:05.853157 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:05.853157 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:05.853157 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:05.854075 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:05.854075 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:05.854075 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:05.854075 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:05.854075 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 371/728] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:05.854644 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed17350-5183-4f02-b604-4a2a1270598d None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:05.854927 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed17350-5183-4f02-b604-4a2a1270598d None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:05.855199 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed17350-5183-4f02-b604-4a2a1270598d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:05.873728 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-6ed17350-5183-4f02-b604-4a2a1270598d None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:05.875088 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-6ed17350-5183-4f02-b604-4a2a1270598d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:05.920981 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3ad96-74e4-42cc-adec-7641004992e8 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:05.947709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 372/729] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:05.949335 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ab1e5-bb2a-4dc8-bbeb-0999b5923a61 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:05.949621 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ab1e5-bb2a-4dc8-bbeb-0999b5923a61 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:05.949895 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6ab1e5-bb2a-4dc8-bbeb-0999b5923a61 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:05.955324 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 373/730] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:05.955324 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ab1e5-bb2a-4dc8-bbeb-0999b5923a61 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:05.968196 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6755ddaf-53b2-484f-bba4-d3064d9e04a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:05.969784 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755ddaf-53b2-484f-bba4-d3064d9e04a1 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:05.970373 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755ddaf-53b2-484f-bba4-d3064d9e04a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:05.970891 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755ddaf-53b2-484f-bba4-d3064d9e04a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:05.972009 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6755ddaf-53b2-484f-bba4-d3064d9e04a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:05.972680 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6755ddaf-53b2-484f-bba4-d3064d9e04a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:05.974553 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6755ddaf-53b2-484f-bba4-d3064d9e04a1 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:05.983499 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 358/731] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 03:10:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:05.985118 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f61a6-8ac5-4129-b729-e22379624584 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:05.985606 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f61a6-8ac5-4129-b729-e22379624584 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:05.986114 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f61a6-8ac5-4129-b729-e22379624584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:06.003169 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-d84f61a6-8ac5-4129-b729-e22379624584 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:06.004454 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-d84f61a6-8ac5-4129-b729-e22379624584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:06.015517 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:06.060028 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6ab1e5-bb2a-4dc8-bbeb-0999b5923a61 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:06.064151 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:06.076626 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 359/732] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:06.077265 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f61a6-8ac5-4129-b729-e22379624584 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:06.096742 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c0ce96-b24e-4377-b70e-107ce2a877d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:06.098163 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c0ce96-b24e-4377-b70e-107ce2a877d6 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:06.098789 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c0ce96-b24e-4377-b70e-107ce2a877d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:06.099248 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c0ce96-b24e-4377-b70e-107ce2a877d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:06.100172 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c0ce96-b24e-4377-b70e-107ce2a877d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:06.100784 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c0ce96-b24e-4377-b70e-107ce2a877d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:06.102359 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c0ce96-b24e-4377-b70e-107ce2a877d6 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:06.115430 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 374/733] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:10:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:06.116400 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c0ce96-b24e-4377-b70e-107ce2a877d6 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:06.183190 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f61a6-8ac5-4129-b729-e22379624584 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:06.218546 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995627ec-4018-4e47-9f77-bec2ebe69ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:06.219217 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995627ec-4018-4e47-9f77-bec2ebe69ea4 None admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:06.219513 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995627ec-4018-4e47-9f77-bec2ebe69ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:06.219759 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995627ec-4018-4e47-9f77-bec2ebe69ea4 None admin] PATH_INFO: `/v3/services` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:06.220523 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995627ec-4018-4e47-9f77-bec2ebe69ea4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:06.221929 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995627ec-4018-4e47-9f77-bec2ebe69ea4 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:06.229637 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c0ce96-b24e-4377-b70e-107ce2a877d6 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:06.245910 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 360/734] 10.222.0.50 () {60 vars in 1141 bytes} [Fri May 27 03:10:06 2022] POST /identity/v3/services => generated 251 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:10:06.246484 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995627ec-4018-4e47-9f77-bec2ebe69ea4 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:06.260536 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:06.288642 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff3951-eadb-4e19-8f98-958bfd914411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:06.290408 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff3951-eadb-4e19-8f98-958bfd914411 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:06.290814 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff3951-eadb-4e19-8f98-958bfd914411 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:06.291226 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdff3951-eadb-4e19-8f98-958bfd914411 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:06.311800 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdff3951-eadb-4e19-8f98-958bfd914411 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:06.313886 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdff3951-eadb-4e19-8f98-958bfd914411 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:06.325731 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:06.343664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-bdff3951-eadb-4e19-8f98-958bfd914411 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:06.343664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:06.343664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:06.343664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:06.343664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:06.343664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:06.343664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:06.343664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:06.343664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:06.343664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:06.343664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:06.343664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:06.343664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:06.343664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:06.343664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:06.344642 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:06.344642 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:06.344642 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:06.344642 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:06.344642 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:06.344642 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:06.344642 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:06.344642 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:06.344642 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:06.344642 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:06.344642 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:06.344642 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:06.344642 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:06.344642 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:06.344642 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:06.344642 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:06.344642 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:06.345701 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:06.345701 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:06.345701 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:06.345701 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:06.345701 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 375/735] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:06.347962 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff3951-eadb-4e19-8f98-958bfd914411 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:06.397172 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995627ec-4018-4e47-9f77-bec2ebe69ea4 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:06.438593 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d365df6-e6e1-448a-9a25-fc4fd809d722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:06.439589 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d365df6-e6e1-448a-9a25-fc4fd809d722 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:06.439970 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d365df6-e6e1-448a-9a25-fc4fd809d722 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:06.440363 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d365df6-e6e1-448a-9a25-fc4fd809d722 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:06.452558 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d365df6-e6e1-448a-9a25-fc4fd809d722 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:06.460576 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-6d365df6-e6e1-448a-9a25-fc4fd809d722 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:10:06.460576 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:06.460576 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:06.460576 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:06.460576 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:06.460576 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:06.460576 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:06.460576 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:06.460576 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:06.460576 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:06.460576 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:06.460576 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:06.460576 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:06.460576 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:06.460576 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:06.460576 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:06.461559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:06.461559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:06.461559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:06.461559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:06.461559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:06.461559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:06.461559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:06.461559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:06.461559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:06.461559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:06.461559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:06.461559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:06.461559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:06.461559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:06.461559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:06.461559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:06.463391 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:06.463391 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:06.463391 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:06.463391 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:06.463391 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:06.463391 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:06.463391 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:06.463391 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:06.463391 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:06.463391 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:06.463391 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:10:06.463391 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:06.463391 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 361/736] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 03:10:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:06.463391 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d365df6-e6e1-448a-9a25-fc4fd809d722 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:06.478741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdff3951-eadb-4e19-8f98-958bfd914411 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:06.521668 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5301821-2d71-4639-bb4d-98974f7c1953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:06.522503 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5301821-2d71-4639-bb4d-98974f7c1953 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:06.522784 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5301821-2d71-4639-bb4d-98974f7c1953 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:06.523060 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5301821-2d71-4639-bb4d-98974f7c1953 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:06.535158 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5301821-2d71-4639-bb4d-98974f7c1953 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:06.538225 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5301821-2d71-4639-bb4d-98974f7c1953 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:06.558758 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-f5301821-2d71-4639-bb4d-98974f7c1953 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 03:10:06.558758 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:06.558758 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:06.558758 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:06.558758 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:06.558758 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:06.558758 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:06.558758 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:06.558758 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:06.558758 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:06.558758 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:06.558758 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:06.558758 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:10:06.558758 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:10:06.558758 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:10:06.558758 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:10:06.559777 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:06.559777 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:06.559777 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:10:06.559777 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:06.559777 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:10:06.559777 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:06.559777 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:06.559777 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:06.559777 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:06.559777 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:06.559777 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:06.559777 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:06.559777 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:06.559777 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:06.559777 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:06.559777 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:06.559777 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:06.559777 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:06.560741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:06.560741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:06.560741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:10:06.560741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:10:06.560741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:10:06.560741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:10:06.560741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:10:06.560741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:10:06.560741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 03:10:06.560741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:06.560741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 376/737] 10.222.0.50 () {56 vars in 1093 bytes} [Fri May 27 03:10:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:06.561965 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5301821-2d71-4639-bb4d-98974f7c1953 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:06.588661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d365df6-e6e1-448a-9a25-fc4fd809d722 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:06.595232 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:06.603361 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:06.628305 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c034b2b7-278a-4c29-94dc-f214b5a71218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:06.629320 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034b2b7-278a-4c29-94dc-f214b5a71218 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:06.629599 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034b2b7-278a-4c29-94dc-f214b5a71218 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:06.629873 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034b2b7-278a-4c29-94dc-f214b5a71218 None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:06.631134 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c034b2b7-278a-4c29-94dc-f214b5a71218 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:06.631549 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c034b2b7-278a-4c29-94dc-f214b5a71218 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:06.633315 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c034b2b7-278a-4c29-94dc-f214b5a71218 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:06.645874 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 362/738] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:10:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:06.647182 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987555a2-0f9c-4d0b-978f-b72411e0219d None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:06.647474 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987555a2-0f9c-4d0b-978f-b72411e0219d None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:06.647743 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987555a2-0f9c-4d0b-978f-b72411e0219d None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:06.652920 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 363/739] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:06.652920 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987555a2-0f9c-4d0b-978f-b72411e0219d None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:06.684214 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5301821-2d71-4639-bb4d-98974f7c1953 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:06.726751 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34fdab-954f-4e80-a9db-6b642433c506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:06.727433 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34fdab-954f-4e80-a9db-6b642433c506 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:06.727716 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34fdab-954f-4e80-a9db-6b642433c506 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:06.727991 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b34fdab-954f-4e80-a9db-6b642433c506 None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:06.728852 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b34fdab-954f-4e80-a9db-6b642433c506 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:06.729055 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b34fdab-954f-4e80-a9db-6b642433c506 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:06.730455 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b34fdab-954f-4e80-a9db-6b642433c506 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:06.739462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 377/740] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:10:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:06.741824 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34fdab-954f-4e80-a9db-6b642433c506 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:06.758508 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987555a2-0f9c-4d0b-978f-b72411e0219d None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:06.797557 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723cd38e-e4ff-4ae9-9514-d809fc62b5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:06.798221 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cd38e-e4ff-4ae9-9514-d809fc62b5af None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:06.798506 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cd38e-e4ff-4ae9-9514-d809fc62b5af None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:06.798782 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cd38e-e4ff-4ae9-9514-d809fc62b5af None admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:06.799630 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723cd38e-e4ff-4ae9-9514-d809fc62b5af None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:06.799935 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723cd38e-e4ff-4ae9-9514-d809fc62b5af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:06.801653 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723cd38e-e4ff-4ae9-9514-d809fc62b5af None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:06.825083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 364/741] 10.222.0.50 () {56 vars in 1133 bytes} [Fri May 27 03:10:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:06.825083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b346de6-4809-44a0-994f-ac46f3e81ab0 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:06.825083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b346de6-4809-44a0-994f-ac46f3e81ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:06.825083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b346de6-4809-44a0-994f-ac46f3e81ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:06.842173 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-2b346de6-4809-44a0-994f-ac46f3e81ab0 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:06.843358 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-2b346de6-4809-44a0-994f-ac46f3e81ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:06.851140 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:06.862845 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b34fdab-954f-4e80-a9db-6b642433c506 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:06.870471 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:06.900188 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b8d900-29f7-49fe-b669-e9d4cd97a480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:06.901039 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8d900-29f7-49fe-b669-e9d4cd97a480 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:06.901319 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8d900-29f7-49fe-b669-e9d4cd97a480 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:06.901755 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8d900-29f7-49fe-b669-e9d4cd97a480 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:06.911350 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b8d900-29f7-49fe-b669-e9d4cd97a480 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:06.912816 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b8d900-29f7-49fe-b669-e9d4cd97a480 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:06.924979 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 365/742] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:06 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 03:10:06.924979 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b346de6-4809-44a0-994f-ac46f3e81ab0 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:06.935063 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-10b8d900-29f7-49fe-b669-e9d4cd97a480 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:10:06.935063 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:06.935063 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:06.935063 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:06.935063 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:06.935063 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:06.935063 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:06.935063 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:06.935063 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:06.935063 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:06.935063 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:06.935063 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:06.935063 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:10:06.935063 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:10:06.935063 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:10:06.935063 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:10:06.936025 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:06.936025 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:06.936025 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:10:06.936025 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:06.936025 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:10:06.936025 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:06.936025 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:06.936025 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:06.936025 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:06.936025 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:06.936025 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:06.936025 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:06.936025 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:06.936025 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:06.936025 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:06.936025 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:06.936025 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:06.936025 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:06.937008 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:06.937008 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:06.937008 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:10:06.937008 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:10:06.937008 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:10:06.937008 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:10:06.937008 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:10:06.937008 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:10:06.937008 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:10:06.937008 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:06.937008 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 378/743] 10.222.0.50 () {56 vars in 1117 bytes} [Fri May 27 03:10:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:06.937607 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b8d900-29f7-49fe-b669-e9d4cd97a480 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:07.040861 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b346de6-4809-44a0-994f-ac46f3e81ab0 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:07.044501 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b8d900-29f7-49fe-b669-e9d4cd97a480 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:07.077159 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb8fad-54b3-4bd1-b533-bc3c039bcab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:07.077880 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb8fad-54b3-4bd1-b533-bc3c039bcab6 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:07.078161 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb8fad-54b3-4bd1-b533-bc3c039bcab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:07.078440 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb8fad-54b3-4bd1-b533-bc3c039bcab6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:07.080480 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ae22d-c38f-4748-944e-d5d1cf0bb641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:07.081306 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ae22d-c38f-4748-944e-d5d1cf0bb641 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:07.081582 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ae22d-c38f-4748-944e-d5d1cf0bb641 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:07.082484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ae22d-c38f-4748-944e-d5d1cf0bb641 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:07.089109 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb8fad-54b3-4bd1-b533-bc3c039bcab6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:07.090592 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb8fad-54b3-4bd1-b533-bc3c039bcab6 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:07.094446 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ae22d-c38f-4748-944e-d5d1cf0bb641 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:07.096097 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ae22d-c38f-4748-944e-d5d1cf0bb641 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:07.109315 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-5cfb8fad-54b3-4bd1-b533-bc3c039bcab6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:07.109315 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:07.109315 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:07.109315 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:07.109315 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:07.109315 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:07.109315 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:07.109315 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:07.109315 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:07.109315 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:07.109315 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:07.109315 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:07.109315 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:07.109315 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:07.109315 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:07.110227 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:07.110227 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:07.110227 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:07.110227 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:07.110227 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:07.110227 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:07.110227 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:07.110227 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:07.110227 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:07.110227 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:07.110227 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:07.110227 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:07.110227 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:07.110227 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:07.110227 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:07.110227 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:07.110227 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:07.111120 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:07.111120 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:07.111120 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:07.111120 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:07.111120 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 366/744] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:07.111383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb8fad-54b3-4bd1-b533-bc3c039bcab6 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:07.118458 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-fe9ae22d-c38f-4748-944e-d5d1cf0bb641 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:07.118458 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:07.118458 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:07.118458 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:07.118458 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:07.118458 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:07.118458 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:07.118458 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:07.118458 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:07.118458 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:07.118458 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:07.118458 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:07.118458 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:07.118458 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:07.118458 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:07.119336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:07.119336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:07.119336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:07.119336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:07.119336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:07.119336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:07.119336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:07.119336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:07.119336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:07.119336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:07.119336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:07.119336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:07.119336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:07.119336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:07.119336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:07.119336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:07.119336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:07.120300 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:07.120300 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:07.120300 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:07.120300 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:07.126592 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 379/745] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:07.127447 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ae22d-c38f-4748-944e-d5d1cf0bb641 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:07.130866 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:07.133971 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:07.227788 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb8fad-54b3-4bd1-b533-bc3c039bcab6 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:07.233344 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ae22d-c38f-4748-944e-d5d1cf0bb641 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:07.264759 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03608218-e7db-4212-978e-86c5351c71e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:07.265674 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03608218-e7db-4212-978e-86c5351c71e4 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:07.266141 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03608218-e7db-4212-978e-86c5351c71e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:07.266406 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03608218-e7db-4212-978e-86c5351c71e4 None admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:07.267237 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03608218-e7db-4212-978e-86c5351c71e4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:07.267540 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03608218-e7db-4212-978e-86c5351c71e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:07.269004 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03608218-e7db-4212-978e-86c5351c71e4 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:07.274402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c9756-a235-422a-930d-5a6e96fdf611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:07.275326 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c9756-a235-422a-930d-5a6e96fdf611 None admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:07.275984 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c9756-a235-422a-930d-5a6e96fdf611 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:07.276571 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c9756-a235-422a-930d-5a6e96fdf611 None admin] PATH_INFO: `/v3/services` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:07.277409 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c9756-a235-422a-930d-5a6e96fdf611 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:07.279818 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c9756-a235-422a-930d-5a6e96fdf611 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:07.289852 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 367/746] 10.222.0.50 () {56 vars in 1149 bytes} [Fri May 27 03:10:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:07.290972 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03608218-e7db-4212-978e-86c5351c71e4 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:07.308979 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 380/747] 10.222.0.50 () {60 vars in 1141 bytes} [Fri May 27 03:10:07 2022] POST /identity/v3/services => generated 240 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:10:07.308979 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c9756-a235-422a-930d-5a6e96fdf611 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:07.372811 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:07.398161 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:07.426812 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03608218-e7db-4212-978e-86c5351c71e4 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:07.456479 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c9756-a235-422a-930d-5a6e96fdf611 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:07.477111 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e812394-ad8b-4e35-b904-6b763084dba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:07.478205 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e812394-ad8b-4e35-b904-6b763084dba4 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:07.478494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e812394-ad8b-4e35-b904-6b763084dba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:07.478768 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e812394-ad8b-4e35-b904-6b763084dba4 None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:07.479561 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e812394-ad8b-4e35-b904-6b763084dba4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:07.479869 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e812394-ad8b-4e35-b904-6b763084dba4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:07.481292 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e812394-ad8b-4e35-b904-6b763084dba4 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:07.498058 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f38eb-4365-41bb-8057-06bb40b6ddf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:07.498420 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 368/748] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:10:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:07.498735 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f38eb-4365-41bb-8057-06bb40b6ddf5 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:07.499016 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f38eb-4365-41bb-8057-06bb40b6ddf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:07.499283 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f38eb-4365-41bb-8057-06bb40b6ddf5 None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:07.499387 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e812394-ad8b-4e35-b904-6b763084dba4 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:07.500007 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f38eb-4365-41bb-8057-06bb40b6ddf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:07.500305 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f38eb-4365-41bb-8057-06bb40b6ddf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:07.501738 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f38eb-4365-41bb-8057-06bb40b6ddf5 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:07.516365 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 381/749] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:10:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:07.518475 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f38eb-4365-41bb-8057-06bb40b6ddf5 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:07.633222 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e812394-ad8b-4e35-b904-6b763084dba4 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:07.641427 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f38eb-4365-41bb-8057-06bb40b6ddf5 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:07.661299 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:07.673657 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:07.681171 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a92393-59ff-4783-bf7d-af4358e0dfbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:07.681520 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164fb632-5933-4da6-99c2-cbb8e1407f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:07.681831 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a92393-59ff-4783-bf7d-af4358e0dfbd None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:07.682109 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a92393-59ff-4783-bf7d-af4358e0dfbd None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:07.682380 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a92393-59ff-4783-bf7d-af4358e0dfbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:07.683251 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fb632-5933-4da6-99c2-cbb8e1407f76 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:07.683718 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fb632-5933-4da6-99c2-cbb8e1407f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:07.684171 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fb632-5933-4da6-99c2-cbb8e1407f76 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:07.692750 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164fb632-5933-4da6-99c2-cbb8e1407f76 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:07.694610 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164fb632-5933-4da6-99c2-cbb8e1407f76 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:07.710448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-164fb632-5933-4da6-99c2-cbb8e1407f76 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 03:10:07.710448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:07.710448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:07.710448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:07.710448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:07.710448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:07.710448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:07.710448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:07.710448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:07.710448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:07.710448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:07.710448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:07.710448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:10:07.710448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:10:07.710448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:10:07.710448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:10:07.711402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:07.711402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:07.711402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:10:07.711402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:07.711402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:10:07.711402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:07.711402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:07.711402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:07.711402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:07.711402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:07.711402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:07.711402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:07.711402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:07.711402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:07.711402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:07.711402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:07.711402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:07.711402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:07.712318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:07.712318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:07.712318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:10:07.712318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:10:07.712318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:10:07.712318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:10:07.712318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:10:07.712318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:10:07.712318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 03:10:07.712318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:07.719228 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 382/750] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 03:10:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:07.720414 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164fb632-5933-4da6-99c2-cbb8e1407f76 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:07.766759 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a92393-59ff-4783-bf7d-af4358e0dfbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:07.768494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a92393-59ff-4783-bf7d-af4358e0dfbd None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:07.845324 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164fb632-5933-4da6-99c2-cbb8e1407f76 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:07.876878 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 369/751] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:10:07 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:10:07.878105 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a92393-59ff-4783-bf7d-af4358e0dfbd None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:07.886785 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b75e557-7875-412a-a688-4d9b36899ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:07.887525 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75e557-7875-412a-a688-4d9b36899ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:07.887861 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75e557-7875-412a-a688-4d9b36899ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:07.888085 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75e557-7875-412a-a688-4d9b36899ef4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:07.895728 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b75e557-7875-412a-a688-4d9b36899ef4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:07.897233 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b75e557-7875-412a-a688-4d9b36899ef4 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:07.911540 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-2b75e557-7875-412a-a688-4d9b36899ef4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:07.911540 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:07.911540 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:07.911540 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:07.911540 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:07.911540 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:07.911540 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:07.911540 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:07.911540 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:07.911540 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:07.911540 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:07.911540 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:07.911540 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:10:07.911540 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:10:07.911540 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:10:07.912448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:10:07.912448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:07.912448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:07.912448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:07.912448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:07.912448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:07.912448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:07.912448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:07.912448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:07.912448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:07.912448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:07.912448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:07.912448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:07.912448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:07.912448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:07.912448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:07.912448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:07.913345 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:10:07.913345 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:10:07.913345 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:10:07.913345 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:10:07.913345 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:10:07.913345 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:10:07.913345 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:07.913345 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:07.913345 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 383/752] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:10:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:07.914340 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b75e557-7875-412a-a688-4d9b36899ef4 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:07.918081 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:07.955537 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:07.999320 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a92393-59ff-4783-bf7d-af4358e0dfbd None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:08.019257 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b75e557-7875-412a-a688-4d9b36899ef4 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:08.037421 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32f0b04-dd9f-4396-aee3-97b7666e7f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:08.038453 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f0b04-dd9f-4396-aee3-97b7666e7f1b None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:08.038944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f0b04-dd9f-4396-aee3-97b7666e7f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:08.039406 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f0b04-dd9f-4396-aee3-97b7666e7f1b None admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:08.040431 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32f0b04-dd9f-4396-aee3-97b7666e7f1b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:08.040976 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32f0b04-dd9f-4396-aee3-97b7666e7f1b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:08.043052 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32f0b04-dd9f-4396-aee3-97b7666e7f1b None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:08.055179 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d10d39-5958-4a85-a8de-e7f2f382ebf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:08.055992 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d10d39-5958-4a85-a8de-e7f2f382ebf5 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:08.056269 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d10d39-5958-4a85-a8de-e7f2f382ebf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:08.056540 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d10d39-5958-4a85-a8de-e7f2f382ebf5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:08.068417 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d10d39-5958-4a85-a8de-e7f2f382ebf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:08.069051 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 370/753] 10.222.0.50 () {56 vars in 1135 bytes} [Fri May 27 03:10:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:08.069677 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d10d39-5958-4a85-a8de-e7f2f382ebf5 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:08.070551 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32f0b04-dd9f-4396-aee3-97b7666e7f1b None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:08.087462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-f0d10d39-5958-4a85-a8de-e7f2f382ebf5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:08.087462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:08.087462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:08.087462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:08.087462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:08.087462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:08.087462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:08.087462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:08.087462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:08.087462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:08.087462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:08.087462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:08.087462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:08.087462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:08.087462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:08.088395 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:08.088395 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:08.088395 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:08.088395 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:08.088395 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:08.088395 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:08.088395 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:08.088395 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:08.088395 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:08.088395 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:08.088395 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:08.088395 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:08.088395 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:08.088395 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:08.088395 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:08.088395 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:08.088395 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:08.089307 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:08.089307 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:08.089307 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:08.089307 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:08.093294 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 384/754] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:08.094156 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d10d39-5958-4a85-a8de-e7f2f382ebf5 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:08.166898 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:08.180562 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32f0b04-dd9f-4396-aee3-97b7666e7f1b None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:08.188931 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:08.200305 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d10d39-5958-4a85-a8de-e7f2f382ebf5 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:08.219182 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584644ca-bbb9-48b1-a86f-b7f40c053368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:08.220214 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584644ca-bbb9-48b1-a86f-b7f40c053368 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:08.220731 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584644ca-bbb9-48b1-a86f-b7f40c053368 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:08.221257 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584644ca-bbb9-48b1-a86f-b7f40c053368 None admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:08.222423 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584644ca-bbb9-48b1-a86f-b7f40c053368 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:08.223085 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584644ca-bbb9-48b1-a86f-b7f40c053368 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:08.224907 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584644ca-bbb9-48b1-a86f-b7f40c053368 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:08.236881 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088f762e-d75e-47b7-a970-524f83336b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:08.237591 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088f762e-d75e-47b7-a970-524f83336b55 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:08.237885 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088f762e-d75e-47b7-a970-524f83336b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:08.238315 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088f762e-d75e-47b7-a970-524f83336b55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:08.240336 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 371/755] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 03:10:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:08.241449 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584644ca-bbb9-48b1-a86f-b7f40c053368 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:08.310199 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088f762e-d75e-47b7-a970-524f83336b55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:08.311559 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088f762e-d75e-47b7-a970-524f83336b55 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:08.354383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584644ca-bbb9-48b1-a86f-b7f40c053368 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:08.389117 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e1a475-7e53-407a-89d7-f45949456018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:08.390200 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e1a475-7e53-407a-89d7-f45949456018 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:08.390725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e1a475-7e53-407a-89d7-f45949456018 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:08.391195 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e1a475-7e53-407a-89d7-f45949456018 None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:08.392148 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e1a475-7e53-407a-89d7-f45949456018 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:08.392671 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e1a475-7e53-407a-89d7-f45949456018 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:08.394441 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e1a475-7e53-407a-89d7-f45949456018 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:08.405955 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:08.411746 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 372/756] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:10:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:08.412977 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e1a475-7e53-407a-89d7-f45949456018 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:08.429427 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 385/757] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:10:08 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:10:08.429427 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088f762e-d75e-47b7-a970-524f83336b55 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:08.430856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:08.527463 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e1a475-7e53-407a-89d7-f45949456018 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:08.530953 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088f762e-d75e-47b7-a970-524f83336b55 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:08.564968 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2203e5d6-a92c-4e6a-9e35-c7051416795d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:08.567297 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203e5d6-a92c-4e6a-9e35-c7051416795d None admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:08.567774 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203e5d6-a92c-4e6a-9e35-c7051416795d None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:08.568137 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397743dc-7811-4175-a15b-9f9a9e1fcbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:08.568425 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203e5d6-a92c-4e6a-9e35-c7051416795d None admin] PATH_INFO: `/v3/projects/f8398941252346468483ea0c863122f4/users/1a699988f08a4001b3b7d764f01f4fc5/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:08.569143 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397743dc-7811-4175-a15b-9f9a9e1fcbae None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:08.569486 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397743dc-7811-4175-a15b-9f9a9e1fcbae None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:08.569820 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397743dc-7811-4175-a15b-9f9a9e1fcbae None admin] PATH_INFO: `/v3/projects/service` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:08.571332 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2203e5d6-a92c-4e6a-9e35-c7051416795d None admin] RBAC: Authorizing `identity:create_grant(project_id=f8398941252346468483ea0c863122f4, user_id=1a699988f08a4001b3b7d764f01f4fc5, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:08.572983 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2203e5d6-a92c-4e6a-9e35-c7051416795d None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:08.577694 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397743dc-7811-4175-a15b-9f9a9e1fcbae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:08.579322 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397743dc-7811-4175-a15b-9f9a9e1fcbae None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:08.593412 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 373/758] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 03:10:08 2022] PUT /identity/v3/projects/f8398941252346468483ea0c863122f4/users/1a699988f08a4001b3b7d764f01f4fc5/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:10:08.594246 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2203e5d6-a92c-4e6a-9e35-c7051416795d None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:08.604943 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-397743dc-7811-4175-a15b-9f9a9e1fcbae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:08.604943 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:08.604943 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:08.604943 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:08.604943 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:08.604943 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:08.604943 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:08.604943 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:08.604943 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:08.604943 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:08.604943 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:08.604943 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:08.604943 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:10:08.604943 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:10:08.604943 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:10:08.605978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:10:08.605978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:08.605978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:08.605978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:08.605978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:08.605978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:08.605978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:08.605978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:08.605978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:08.605978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:08.605978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:08.605978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:08.605978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:08.605978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:08.605978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:08.605978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:08.605978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:08.606862 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:10:08.606862 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:10:08.606862 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:10:08.606862 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:10:08.606862 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:10:08.606862 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:10:08.606862 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:08.606862 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:08.610042 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 386/759] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:10:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:08.615076 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df284166-6a4c-4c99-ba11-80135e810691 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:08.615819 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df284166-6a4c-4c99-ba11-80135e810691 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:08.616273 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df284166-6a4c-4c99-ba11-80135e810691 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:08.620558 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 387/760] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:08.621822 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df284166-6a4c-4c99-ba11-80135e810691 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:08.678337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:08.685271 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:08.719451 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2203e5d6-a92c-4e6a-9e35-c7051416795d None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:08.748393 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df284166-6a4c-4c99-ba11-80135e810691 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:08.757094 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3554592-2283-45e3-abc7-8717945360ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:08.758588 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3554592-2283-45e3-abc7-8717945360ca None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:08.758588 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3554592-2283-45e3-abc7-8717945360ca None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:08.758783 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3554592-2283-45e3-abc7-8717945360ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:08.770284 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3554592-2283-45e3-abc7-8717945360ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:08.771899 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3554592-2283-45e3-abc7-8717945360ca None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:08.789979 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc204b50-a8bf-4263-b031-302f6c0e6a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:08.791003 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc204b50-a8bf-4263-b031-302f6c0e6a1e None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:08.792042 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc204b50-a8bf-4263-b031-302f6c0e6a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:08.792152 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-d3554592-2283-45e3-abc7-8717945360ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:08.792152 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:08.792152 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:08.792152 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:08.792152 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:08.792152 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:08.792152 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:08.792152 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:08.792152 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:08.792152 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:08.792152 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:08.792152 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:08.792152 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:08.792152 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:08.792152 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:08.792985 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:08.792985 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:08.792985 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:08.792985 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:08.792985 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:08.792985 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:08.792985 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:08.792985 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:08.792985 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:08.792985 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:08.792985 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:08.792985 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:08.792985 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:08.792985 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:08.792985 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:08.792985 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:08.792985 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:08.793891 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:08.793891 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:08.793891 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:08.793891 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:08.793891 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 374/761] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:08.794942 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc204b50-a8bf-4263-b031-302f6c0e6a1e None admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:08.795580 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9482d9-511b-415d-8ab1-34bd82b08961 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:08.795922 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9482d9-511b-415d-8ab1-34bd82b08961 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:08.796258 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9482d9-511b-415d-8ab1-34bd82b08961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:08.796464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc204b50-a8bf-4263-b031-302f6c0e6a1e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:08.796968 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc204b50-a8bf-4263-b031-302f6c0e6a1e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:08.798588 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc204b50-a8bf-4263-b031-302f6c0e6a1e None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:08.813830 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 388/762] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 03:10:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:08.814979 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc204b50-a8bf-4263-b031-302f6c0e6a1e None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:08.820050 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-4d9482d9-511b-415d-8ab1-34bd82b08961 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:08.821461 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-4d9482d9-511b-415d-8ab1-34bd82b08961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:08.901104 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 375/763] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:10:08.902998 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9482d9-511b-415d-8ab1-34bd82b08961 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:08.938787 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc204b50-a8bf-4263-b031-302f6c0e6a1e None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:08.954503 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:08.974870 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:08.984284 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd625bde-c1fc-4ecd-b524-d18a66d89672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:08.985358 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd625bde-c1fc-4ecd-b524-d18a66d89672 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:08.985779 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd625bde-c1fc-4ecd-b524-d18a66d89672 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:08.986209 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd625bde-c1fc-4ecd-b524-d18a66d89672 None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:08.987082 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd625bde-c1fc-4ecd-b524-d18a66d89672 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:08.987613 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd625bde-c1fc-4ecd-b524-d18a66d89672 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:08.989466 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd625bde-c1fc-4ecd-b524-d18a66d89672 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:09.002534 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 389/764] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:10:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:09.008180 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd625bde-c1fc-4ecd-b524-d18a66d89672 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:09.029762 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9482d9-511b-415d-8ab1-34bd82b08961 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:09.068661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b665279-322d-448c-8e33-acc6b8c51320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:09.069343 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b665279-322d-448c-8e33-acc6b8c51320 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:09.069527 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b665279-322d-448c-8e33-acc6b8c51320 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:09.069843 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b665279-322d-448c-8e33-acc6b8c51320 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:09.070688 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b665279-322d-448c-8e33-acc6b8c51320 None admin] RBAC: Adding query filter params (role.id=3451aa51e64642768b2228126d20fa03, scope.project.id=f8398941252346468483ea0c863122f4, user.id=86a18e313e4944748aef58d54bfe7b13) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:09.070999 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b665279-322d-448c-8e33-acc6b8c51320 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:09.072552 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b665279-322d-448c-8e33-acc6b8c51320 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:09.079050 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 376/765] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:10:08 2022] GET /identity/v3/role_assignments?user.id=86a18e313e4944748aef58d54bfe7b13&scope.project.id=f8398941252346468483ea0c863122f4&role.id=3451aa51e64642768b2228126d20fa03 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:09.080074 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b665279-322d-448c-8e33-acc6b8c51320 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:09.118998 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd625bde-c1fc-4ecd-b524-d18a66d89672 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:09.156701 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a4605-feff-4200-8673-9575f7310244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:09.157940 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a4605-feff-4200-8673-9575f7310244 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:09.158453 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a4605-feff-4200-8673-9575f7310244 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:09.159279 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a4605-feff-4200-8673-9575f7310244 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:09.160496 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a4605-feff-4200-8673-9575f7310244 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:09.162393 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a4605-feff-4200-8673-9575f7310244 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:09.178468 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-b55a4605-feff-4200-8673-9575f7310244 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 03:10:09.178468 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:09.178468 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:10:09.178468 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:10:09.178468 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:10:09.178468 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:10:09.178468 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:10:09.178468 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:09.178468 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 03:10:09.178468 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:09.178468 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:10:09.178468 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:09.178468 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:09.178468 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:09.178468 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:09.178468 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:09.179504 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:09.179504 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:09.179504 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:09.179504 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:09.179504 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:09.179504 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:09.179504 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:09.179504 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:10:09.179504 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:10:09.179504 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:10:09.179504 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:10:09.179504 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:09.179504 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:09.179504 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:09.179504 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:09.179504 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:09.180397 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:09.180397 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:09.180397 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:09.180397 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:09.180397 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:09.180397 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:09.180397 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:09.180397 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:09.180397 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:09.180397 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:09.180397 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:09.180397 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:10:09.180397 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:09.180397 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 03:10:09.180397 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:09.183720 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 390/766] 10.222.0.50 () {56 vars in 1129 bytes} [Fri May 27 03:10:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:09.191406 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a4605-feff-4200-8673-9575f7310244 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:09.215744 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b665279-322d-448c-8e33-acc6b8c51320 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:09.231410 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:09.234291 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:09.257902 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb76273-6b03-4ee6-bd5d-48104dec6427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:09.258824 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb76273-6b03-4ee6-bd5d-48104dec6427 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:09.259171 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb76273-6b03-4ee6-bd5d-48104dec6427 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:09.259511 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb76273-6b03-4ee6-bd5d-48104dec6427 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:09.281397 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb76273-6b03-4ee6-bd5d-48104dec6427 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:09.283161 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb76273-6b03-4ee6-bd5d-48104dec6427 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:09.312983 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-1fb76273-6b03-4ee6-bd5d-48104dec6427 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:09.312983 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:09.312983 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:09.312983 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:09.312983 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:09.312983 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:09.312983 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:09.312983 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:09.312983 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:09.312983 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:09.312983 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:09.312983 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:09.312983 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:10:09.312983 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:10:09.312983 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:10:09.313971 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:10:09.313971 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:09.313971 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:09.313971 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:09.313971 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:09.313971 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:09.313971 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:09.313971 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:09.313971 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:09.313971 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:09.313971 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:09.313971 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:09.313971 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:09.313971 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:09.313971 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:09.313971 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:09.313971 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:09.314846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:10:09.314846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:10:09.314846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:10:09.314846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:10:09.314846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:10:09.314846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:10:09.314846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:09.314846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:09.314846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 377/767] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:10:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:09.322641 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb76273-6b03-4ee6-bd5d-48104dec6427 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:09.324029 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a4605-feff-4200-8673-9575f7310244 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:09.361126 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d97ed-6f8c-4509-91ac-286fb615f582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:09.362243 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d97ed-6f8c-4509-91ac-286fb615f582 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:09.362882 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d97ed-6f8c-4509-91ac-286fb615f582 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:09.363403 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d97ed-6f8c-4509-91ac-286fb615f582 None admin] PATH_INFO: `/v3/services` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:09.364436 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d97ed-6f8c-4509-91ac-286fb615f582 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:09.364928 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d97ed-6f8c-4509-91ac-286fb615f582 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:09.366772 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d97ed-6f8c-4509-91ac-286fb615f582 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:09.372336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 391/768] 10.222.0.50 () {56 vars in 1124 bytes} [Fri May 27 03:10:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:09.378184 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d97ed-6f8c-4509-91ac-286fb615f582 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:09.443262 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb76273-6b03-4ee6-bd5d-48104dec6427 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:09.480901 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7e8c3-b1c8-4f23-a80b-59750d214979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:09.481660 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7e8c3-b1c8-4f23-a80b-59750d214979 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:09.481971 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7e8c3-b1c8-4f23-a80b-59750d214979 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:09.482348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7e8c3-b1c8-4f23-a80b-59750d214979 None admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:09.483145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7e8c3-b1c8-4f23-a80b-59750d214979 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:09.483452 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7e8c3-b1c8-4f23-a80b-59750d214979 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:09.484890 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7e8c3-b1c8-4f23-a80b-59750d214979 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:09.491921 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:09.496479 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d97ed-6f8c-4509-91ac-286fb615f582 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:09.504993 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 378/769] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 03:10:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:09.510166 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7e8c3-b1c8-4f23-a80b-59750d214979 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:09.568070 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:09.576872 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f88df1-c123-4225-bfdc-0bd147b3199f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:09.577603 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f88df1-c123-4225-bfdc-0bd147b3199f None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:09.577916 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f88df1-c123-4225-bfdc-0bd147b3199f None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:09.578151 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f88df1-c123-4225-bfdc-0bd147b3199f None admin] PATH_INFO: `/v3/services` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:09.578867 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f88df1-c123-4225-bfdc-0bd147b3199f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:09.579187 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f88df1-c123-4225-bfdc-0bd147b3199f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:09.580583 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f88df1-c123-4225-bfdc-0bd147b3199f None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:09.590215 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 392/770] 10.222.0.50 () {56 vars in 1124 bytes} [Fri May 27 03:10:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:09.599885 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f88df1-c123-4225-bfdc-0bd147b3199f None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:09.671245 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7e8c3-b1c8-4f23-a80b-59750d214979 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:09.717380 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f88df1-c123-4225-bfdc-0bd147b3199f None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:09.717905 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ea0d9-66cf-4846-9f34-ba6edbee46bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:09.719032 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ea0d9-66cf-4846-9f34-ba6edbee46bc None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:09.720383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ea0d9-66cf-4846-9f34-ba6edbee46bc None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:09.720877 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ea0d9-66cf-4846-9f34-ba6edbee46bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:09.721997 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3ea0d9-66cf-4846-9f34-ba6edbee46bc None admin] RBAC: Adding query filter params (role.id=b4c524dd7bb44e94bd3b42d86e1cdf42, scope.project.id=f8398941252346468483ea0c863122f4, user.id=3de53e2f0fba4e0fac70887067c1963d) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:09.722567 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3ea0d9-66cf-4846-9f34-ba6edbee46bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:09.724244 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3ea0d9-66cf-4846-9f34-ba6edbee46bc None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:09.737146 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 379/771] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:10:09 2022] GET /identity/v3/role_assignments?user.id=3de53e2f0fba4e0fac70887067c1963d&scope.project.id=f8398941252346468483ea0c863122f4&role.id=b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:09.737146 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae74b23-6328-43b2-8529-daf2a8681ec7 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:09.737146 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae74b23-6328-43b2-8529-daf2a8681ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:09.737146 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae74b23-6328-43b2-8529-daf2a8681ec7 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:09.740868 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 380/772] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:09.748990 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9325b-3eee-4e2c-b6b9-feb6745e9322 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:09.748990 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9325b-3eee-4e2c-b6b9-feb6745e9322 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:09.748990 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9325b-3eee-4e2c-b6b9-feb6745e9322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:09.780409 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-d7e9325b-3eee-4e2c-b6b9-feb6745e9322 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:09.781637 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-d7e9325b-3eee-4e2c-b6b9-feb6745e9322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:09.790565 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8362e17f-04f9-405e-a865-0221dc77d473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:09.791252 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362e17f-04f9-405e-a865-0221dc77d473 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:09.791526 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362e17f-04f9-405e-a865-0221dc77d473 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:09.791796 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362e17f-04f9-405e-a865-0221dc77d473 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:09.792590 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8362e17f-04f9-405e-a865-0221dc77d473 None admin] RBAC: Adding query filter params (interface=public, service_id=22f7088173bd47ca8bf3adf15dffb772, region_id=RegionOne) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:09.792899 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8362e17f-04f9-405e-a865-0221dc77d473 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:09.794324 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8362e17f-04f9-405e-a865-0221dc77d473 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:09.811240 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 393/773] 10.222.0.50 () {56 vars in 1251 bytes} [Fri May 27 03:10:09 2022] GET /identity/v3/endpoints?service_id=22f7088173bd47ca8bf3adf15dffb772&interface=public®ion_id=RegionOne => generated 194 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:09.815357 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8362e17f-04f9-405e-a865-0221dc77d473 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:09.861408 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:09.938517 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 381/774] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:10:09.966965 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e9325b-3eee-4e2c-b6b9-feb6745e9322 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:09.977284 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8362e17f-04f9-405e-a865-0221dc77d473 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:10.001046 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:10.038216 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d7696-6d90-4bd2-b514-9c93b0547b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:10.038945 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d7696-6d90-4bd2-b514-9c93b0547b43 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:10.039182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d7696-6d90-4bd2-b514-9c93b0547b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:10.039450 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d7696-6d90-4bd2-b514-9c93b0547b43 None admin] PATH_INFO: `/v3/services` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:10.040145 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374d7696-6d90-4bd2-b514-9c93b0547b43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:10.041606 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374d7696-6d90-4bd2-b514-9c93b0547b43 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:10.059981 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 394/775] 10.222.0.50 () {56 vars in 1087 bytes} [Fri May 27 03:10:09 2022] GET /identity/v3/services => generated 772 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:10.092581 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e9325b-3eee-4e2c-b6b9-feb6745e9322 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:10.149210 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff9ad1-989b-46ec-9be1-d5c0673427ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:10.150029 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff9ad1-989b-46ec-9be1-d5c0673427ba None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:10.150304 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff9ad1-989b-46ec-9be1-d5c0673427ba None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:10.150573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff9ad1-989b-46ec-9be1-d5c0673427ba None admin] PATH_INFO: `/v3/services/network` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:10.151240 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff9ad1-989b-46ec-9be1-d5c0673427ba None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:10.152745 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff9ad1-989b-46ec-9be1-d5c0673427ba None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:10.167778 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-e5ff9ad1-989b-46ec-9be1-d5c0673427ba None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 27 03:10:10.167778 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:10.167778 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:10:10.167778 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:10:10.167778 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:10:10.167778 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:10:10.167778 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:10:10.167778 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:10.167778 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 03:10:10.167778 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:10.167778 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:10:10.167778 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:10.167778 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:10.167778 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:10.167778 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:10.167778 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:10.167778 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:10.168818 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:10.168818 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:10.168818 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:10.168818 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:10.168818 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:10.168818 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:10.168818 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:10:10.168818 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:10:10.168818 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:10:10.168818 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:10:10.168818 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:10.168818 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:10.168818 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:10.168818 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:10.168818 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:10.168818 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:10.169662 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:10.169662 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:10.169662 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:10.169662 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:10.169662 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:10.169662 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:10.169662 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:10.169662 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:10.169662 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:10.169662 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:10.169662 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:10:10.169662 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:10.169662 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 03:10:10.169662 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:10.173292 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 382/776] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:10:09 2022] GET /identity/v3/services/network => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:10.175572 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d7696-6d90-4bd2-b514-9c93b0547b43 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:10.451807 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:10.465090 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d7696-6d90-4bd2-b514-9c93b0547b43 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:10.510796 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be14817-a923-4f3c-8e48-34647b062dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:10.511479 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be14817-a923-4f3c-8e48-34647b062dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:10.511879 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be14817-a923-4f3c-8e48-34647b062dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:10.512152 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be14817-a923-4f3c-8e48-34647b062dc3 None admin] PATH_INFO: `/v3/services` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:10.512871 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be14817-a923-4f3c-8e48-34647b062dc3 None admin] RBAC: Adding query filter params (name=network) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:10.513192 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be14817-a923-4f3c-8e48-34647b062dc3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:10.514588 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be14817-a923-4f3c-8e48-34647b062dc3 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:10.520064 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 395/777] 10.222.0.50 () {56 vars in 1112 bytes} [Fri May 27 03:10:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:10.525643 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff9ad1-989b-46ec-9be1-d5c0673427ba None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:10.668808 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:10.685122 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff9ad1-989b-46ec-9be1-d5c0673427ba None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:10.730960 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a2e349-4340-413b-9350-fd7d11d78b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:10.731941 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2e349-4340-413b-9350-fd7d11d78b72 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:10.732219 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2e349-4340-413b-9350-fd7d11d78b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:10.732486 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2e349-4340-413b-9350-fd7d11d78b72 None admin] PATH_INFO: `/v3/services` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:10.733223 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a2e349-4340-413b-9350-fd7d11d78b72 None admin] RBAC: Adding query filter params (type=network) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:10.733520 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a2e349-4340-413b-9350-fd7d11d78b72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:10.734895 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a2e349-4340-413b-9350-fd7d11d78b72 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:10.746315 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 383/778] 10.222.0.50 () {56 vars in 1112 bytes} [Fri May 27 03:10:10 2022] GET /identity/v3/services?type=network => generated 350 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:10.752541 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be14817-a923-4f3c-8e48-34647b062dc3 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:10.892245 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be14817-a923-4f3c-8e48-34647b062dc3 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:10.946586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77285186-e586-4579-b1d9-12dc4b2e2c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:10.947267 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77285186-e586-4579-b1d9-12dc4b2e2c17 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:10.947547 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77285186-e586-4579-b1d9-12dc4b2e2c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:10.947818 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77285186-e586-4579-b1d9-12dc4b2e2c17 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:10.948613 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77285186-e586-4579-b1d9-12dc4b2e2c17 None admin] RBAC: Adding query filter params (interface=public, service_id=ee8e5e71d64d4d989b7e38df5d925aec, region_id=RegionOne) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:10.948919 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77285186-e586-4579-b1d9-12dc4b2e2c17 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:10.950370 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77285186-e586-4579-b1d9-12dc4b2e2c17 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:10.961659 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 396/779] 10.222.0.50 () {56 vars in 1251 bytes} [Fri May 27 03:10:10 2022] GET /identity/v3/endpoints?service_id=ee8e5e71d64d4d989b7e38df5d925aec&interface=public®ion_id=RegionOne => generated 194 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:10.966942 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a2e349-4340-413b-9350-fd7d11d78b72 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:11.113792 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a2e349-4340-413b-9350-fd7d11d78b72 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:11.149815 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c6a853-761d-489c-b4fe-73d04459e29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:11.150948 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6a853-761d-489c-b4fe-73d04459e29c None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:11.151268 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6a853-761d-489c-b4fe-73d04459e29c None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:11.151544 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6a853-761d-489c-b4fe-73d04459e29c None admin] PATH_INFO: `/v3/services` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:11.152387 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c6a853-761d-489c-b4fe-73d04459e29c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:11.154364 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c6a853-761d-489c-b4fe-73d04459e29c None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:11.163060 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 384/780] 10.222.0.50 () {56 vars in 1087 bytes} [Fri May 27 03:10:10 2022] GET /identity/v3/services => generated 772 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:11.407677 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091cd6c0-bb4a-4cb0-84d7-f2d93db9d6f6 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:11.408261 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091cd6c0-bb4a-4cb0-84d7-f2d93db9d6f6 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:11.408261 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091cd6c0-bb4a-4cb0-84d7-f2d93db9d6f6 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:11.410448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 397/781] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:11.416609 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25bca83-0364-410e-8a26-337f01b64d48 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:11.416933 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25bca83-0364-410e-8a26-337f01b64d48 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:11.417220 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25bca83-0364-410e-8a26-337f01b64d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:11.437680 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-f25bca83-0364-410e-8a26-337f01b64d48 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:11.438898 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-f25bca83-0364-410e-8a26-337f01b64d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:11.455873 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:11.529133 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 385/782] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:10:11.564224 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091cd6c0-bb4a-4cb0-84d7-f2d93db9d6f6 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:11.592384 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:11.703012 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091cd6c0-bb4a-4cb0-84d7-f2d93db9d6f6 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:11.738383 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c4370-0ca0-4ee7-82bd-3ee6ba69cbf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:11.739273 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c4370-0ca0-4ee7-82bd-3ee6ba69cbf6 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:11.739556 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c4370-0ca0-4ee7-82bd-3ee6ba69cbf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:11.739834 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c4370-0ca0-4ee7-82bd-3ee6ba69cbf6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:11.755042 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c4370-0ca0-4ee7-82bd-3ee6ba69cbf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:11.763101 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-4d0c4370-0ca0-4ee7-82bd-3ee6ba69cbf6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:10:11.763101 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:11.763101 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:11.763101 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:11.763101 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:11.763101 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:11.763101 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:11.763101 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:11.763101 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:11.763101 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:11.763101 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:11.763101 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:11.763101 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:11.763101 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:11.763101 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:11.763101 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:11.764012 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:11.764012 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:11.764012 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:11.764012 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:11.764012 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:11.764012 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:11.764012 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:11.764012 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:11.764012 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:11.764012 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:11.764012 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:11.764012 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:11.764012 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:11.764012 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:11.764012 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:11.764012 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:11.764850 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:11.764850 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:11.764850 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:11.764850 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:11.764850 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:11.764850 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:11.764850 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:11.764850 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:11.764850 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:11.764850 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:11.764850 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:10:11.764850 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:11.764850 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 398/783] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 03:10:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:11.774094 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25bca83-0364-410e-8a26-337f01b64d48 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:11.794467 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbab261-4071-48b7-b2ac-afbbaab73f91 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:11.794803 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbab261-4071-48b7-b2ac-afbbaab73f91 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:11.795012 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbab261-4071-48b7-b2ac-afbbaab73f91 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:11.796644 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 399/784] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:11.802015 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d176a57-9f28-40c1-84ab-8e7ad8f76894 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:11.802298 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d176a57-9f28-40c1-84ab-8e7ad8f76894 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:11.802575 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d176a57-9f28-40c1-84ab-8e7ad8f76894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:11.827922 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-2d176a57-9f28-40c1-84ab-8e7ad8f76894 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:11.829090 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-2d176a57-9f28-40c1-84ab-8e7ad8f76894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:11.950789 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 400/785] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:10:11.968009 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25bca83-0364-410e-8a26-337f01b64d48 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:11.972925 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d176a57-9f28-40c1-84ab-8e7ad8f76894 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:11.989357 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:12.045082 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501130e-c6b5-445b-8b95-9ac35576209d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:12.045769 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501130e-c6b5-445b-8b95-9ac35576209d None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:12.046047 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501130e-c6b5-445b-8b95-9ac35576209d None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:12.046319 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501130e-c6b5-445b-8b95-9ac35576209d None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:12.047083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d501130e-c6b5-445b-8b95-9ac35576209d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:12.047388 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d501130e-c6b5-445b-8b95-9ac35576209d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:12.048780 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d501130e-c6b5-445b-8b95-9ac35576209d None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:12.058989 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 386/786] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:10:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:12.066294 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501130e-c6b5-445b-8b95-9ac35576209d None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:12.084452 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:12.131043 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d176a57-9f28-40c1-84ab-8e7ad8f76894 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:12.201592 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0aad50-6013-45d3-81d1-7b32b1403550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:12.202405 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0aad50-6013-45d3-81d1-7b32b1403550 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:12.202683 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0aad50-6013-45d3-81d1-7b32b1403550 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:12.202958 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0aad50-6013-45d3-81d1-7b32b1403550 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:12.220445 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501130e-c6b5-445b-8b95-9ac35576209d None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:12.221512 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0aad50-6013-45d3-81d1-7b32b1403550 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:12.238333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-7e0aad50-6013-45d3-81d1-7b32b1403550 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:10:12.238333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:12.238333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:12.238333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:12.238333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:12.238333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:12.238333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:12.238333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:12.238333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:12.238333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:12.238333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:12.238333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:12.238333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:12.238333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:12.238333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:12.238333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:12.239489 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:12.239489 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:12.239489 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:12.239489 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:12.239489 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:12.239489 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:12.239489 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:12.239489 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:12.239489 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:12.239489 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:12.239489 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:12.239489 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:12.239489 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:12.239489 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:12.239489 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:12.239489 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:12.240314 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:12.240314 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:12.240314 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:12.240314 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:12.240314 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:12.240314 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:12.240314 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:12.240314 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:12.240314 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:12.240314 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:12.240314 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:10:12.240314 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:12.240314 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 401/787] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 03:10:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:10:12.249860 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0aad50-6013-45d3-81d1-7b32b1403550 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:12.256482 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2308c606-f4d2-48a9-85cb-1d47a06394ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:12.257363 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308c606-f4d2-48a9-85cb-1d47a06394ce None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:12.257645 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308c606-f4d2-48a9-85cb-1d47a06394ce None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:12.257919 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2308c606-f4d2-48a9-85cb-1d47a06394ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:12.279231 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2308c606-f4d2-48a9-85cb-1d47a06394ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:12.280711 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2308c606-f4d2-48a9-85cb-1d47a06394ce None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:12.323738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-2308c606-f4d2-48a9-85cb-1d47a06394ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:12.323738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:12.323738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:12.323738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:12.323738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:12.323738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:12.323738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:12.323738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:12.323738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:12.323738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:12.323738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:12.323738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:12.323738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:12.323738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:12.323738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:12.326145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:12.326145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:12.326145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:12.326145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:12.326145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:12.326145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:12.326145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:12.326145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:12.326145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:12.326145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:12.326145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:12.326145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:12.326145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:12.326145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:12.326145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:12.326145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:12.326145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:12.328139 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:12.328139 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:12.328139 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:12.328139 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:12.328139 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 387/788] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:12.336056 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2308c606-f4d2-48a9-85cb-1d47a06394ce None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:12.368320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:12.403580 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:12.407199 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0aad50-6013-45d3-81d1-7b32b1403550 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:12.445173 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d879665b-d7d4-491a-996c-e5dcb88bd2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:12.445972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d879665b-d7d4-491a-996c-e5dcb88bd2d6 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:12.446210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d879665b-d7d4-491a-996c-e5dcb88bd2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:12.446488 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d879665b-d7d4-491a-996c-e5dcb88bd2d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:12.447350 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d879665b-d7d4-491a-996c-e5dcb88bd2d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:12.447659 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d879665b-d7d4-491a-996c-e5dcb88bd2d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:12.449188 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d879665b-d7d4-491a-996c-e5dcb88bd2d6 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:12.458493 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 402/789] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 03:10:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:12.464263 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d879665b-d7d4-491a-996c-e5dcb88bd2d6 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:12.473475 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2308c606-f4d2-48a9-85cb-1d47a06394ce None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:12.511744 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0629e27-8fe9-4a0d-bd20-5b1b38b90de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:12.512695 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0629e27-8fe9-4a0d-bd20-5b1b38b90de0 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:12.513290 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0629e27-8fe9-4a0d-bd20-5b1b38b90de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:12.513707 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0629e27-8fe9-4a0d-bd20-5b1b38b90de0 None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:12.514735 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0629e27-8fe9-4a0d-bd20-5b1b38b90de0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:12.515235 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0629e27-8fe9-4a0d-bd20-5b1b38b90de0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:12.516918 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0629e27-8fe9-4a0d-bd20-5b1b38b90de0 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:12.530891 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 388/790] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:10:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:12.537626 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0629e27-8fe9-4a0d-bd20-5b1b38b90de0 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:12.577837 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d879665b-d7d4-491a-996c-e5dcb88bd2d6 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:12.612574 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:12.622491 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275bbf35-6950-4a66-8156-c19ba70940ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:12.623309 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275bbf35-6950-4a66-8156-c19ba70940ff None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:12.623594 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275bbf35-6950-4a66-8156-c19ba70940ff None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:12.623931 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275bbf35-6950-4a66-8156-c19ba70940ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:12.642914 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275bbf35-6950-4a66-8156-c19ba70940ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:12.644800 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275bbf35-6950-4a66-8156-c19ba70940ff None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:12.675873 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-275bbf35-6950-4a66-8156-c19ba70940ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:12.675873 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:12.675873 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:12.675873 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:12.675873 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:12.675873 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:12.675873 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:12.675873 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:12.675873 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:12.675873 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:12.675873 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:12.675873 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:12.675873 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:12.675873 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:12.675873 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:12.676845 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:12.676845 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:12.676845 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:12.676845 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:12.676845 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:12.676845 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:12.676845 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:12.676845 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:12.676845 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:12.676845 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:12.676845 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:12.676845 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:12.676845 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:12.676845 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:12.676845 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:12.676845 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:12.676845 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:12.677748 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:12.677748 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:12.677748 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:12.677748 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:12.683474 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 403/791] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:12.692007 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275bbf35-6950-4a66-8156-c19ba70940ff None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:12.709643 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0629e27-8fe9-4a0d-bd20-5b1b38b90de0 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:12.741979 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:12.756595 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b1f20-4b85-48be-85bb-127a40198111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:12.757697 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b1f20-4b85-48be-85bb-127a40198111 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:12.757984 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b1f20-4b85-48be-85bb-127a40198111 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:12.758316 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b1f20-4b85-48be-85bb-127a40198111 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:12.768270 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b1f20-4b85-48be-85bb-127a40198111 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:12.770056 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b1f20-4b85-48be-85bb-127a40198111 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:12.789969 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-f95b1f20-4b85-48be-85bb-127a40198111 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 03:10:12.789969 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:12.789969 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:12.789969 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:12.789969 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:12.789969 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:12.789969 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:12.789969 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:12.789969 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:12.789969 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:12.789969 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:12.789969 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:12.789969 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:10:12.789969 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:10:12.789969 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:10:12.789969 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:10:12.790944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:12.790944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:12.790944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:10:12.790944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:12.790944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:10:12.790944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:12.790944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:12.790944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:12.790944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:12.790944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:12.790944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:12.790944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:12.790944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:12.790944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:12.790944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:12.790944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:12.790944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:12.790944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:12.791863 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:12.791863 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:12.791863 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:10:12.791863 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:10:12.791863 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:10:12.791863 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:10:12.791863 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:10:12.791863 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:10:12.791863 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 03:10:12.791863 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:12.796479 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 389/792] 10.222.0.50 () {56 vars in 1093 bytes} [Fri May 27 03:10:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:12.798100 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5656d-c439-408b-9713-56fead1ac88b None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:12.798393 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5656d-c439-408b-9713-56fead1ac88b None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:12.798674 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5656d-c439-408b-9713-56fead1ac88b None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:12.801257 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 390/793] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:12.802583 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d5656d-c439-408b-9713-56fead1ac88b None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:12.810355 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275bbf35-6950-4a66-8156-c19ba70940ff None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:12.851475 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f5ed49-1d7e-481d-a25d-20030b06f2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:12.852597 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f5ed49-1d7e-481d-a25d-20030b06f2df None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:12.853102 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f5ed49-1d7e-481d-a25d-20030b06f2df None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:12.853563 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f5ed49-1d7e-481d-a25d-20030b06f2df None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:12.854491 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f5ed49-1d7e-481d-a25d-20030b06f2df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:12.854997 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f5ed49-1d7e-481d-a25d-20030b06f2df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:12.856565 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f5ed49-1d7e-481d-a25d-20030b06f2df None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:12.870440 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 404/794] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:10:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:12.872262 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb380ead-9f6e-4522-9286-3f065eb15b64 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:12.872742 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb380ead-9f6e-4522-9286-3f065eb15b64 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:12.873229 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb380ead-9f6e-4522-9286-3f065eb15b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:12.895950 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-cb380ead-9f6e-4522-9286-3f065eb15b64 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:12.897479 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-cb380ead-9f6e-4522-9286-3f065eb15b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:12.942077 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d5656d-c439-408b-9713-56fead1ac88b None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:12.942285 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:12.993125 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 405/795] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:10:12.994781 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb380ead-9f6e-4522-9286-3f065eb15b64 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:12.999579 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294af6d8-8ce8-4a00-85e9-b09dda5cccd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:13.000314 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294af6d8-8ce8-4a00-85e9-b09dda5cccd1 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:13.000598 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294af6d8-8ce8-4a00-85e9-b09dda5cccd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:13.000873 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294af6d8-8ce8-4a00-85e9-b09dda5cccd1 None admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:13.001816 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294af6d8-8ce8-4a00-85e9-b09dda5cccd1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:13.002126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294af6d8-8ce8-4a00-85e9-b09dda5cccd1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:13.003865 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294af6d8-8ce8-4a00-85e9-b09dda5cccd1 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:13.027392 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 391/796] 10.222.0.50 () {56 vars in 1133 bytes} [Fri May 27 03:10:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:13.028466 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294af6d8-8ce8-4a00-85e9-b09dda5cccd1 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:13.078154 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:13.114809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb380ead-9f6e-4522-9286-3f065eb15b64 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:13.140278 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294af6d8-8ce8-4a00-85e9-b09dda5cccd1 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:13.150185 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3e9aa-24e9-4e22-be76-e0fcec655380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:13.151122 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3e9aa-24e9-4e22-be76-e0fcec655380 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:13.151397 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3e9aa-24e9-4e22-be76-e0fcec655380 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:13.151669 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed3e9aa-24e9-4e22-be76-e0fcec655380 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:13.159557 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed3e9aa-24e9-4e22-be76-e0fcec655380 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:13.161037 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed3e9aa-24e9-4e22-be76-e0fcec655380 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:13.175828 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-1ed3e9aa-24e9-4e22-be76-e0fcec655380 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:10:13.175828 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:13.175828 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:13.175828 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:13.175828 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:13.175828 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:13.175828 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:13.175828 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:13.175828 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:13.175828 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:13.175828 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:13.175828 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:13.175828 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:10:13.175828 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:10:13.175828 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:10:13.175828 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:10:13.176747 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:13.176747 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:13.176747 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:10:13.176747 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:13.176747 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:10:13.176747 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:13.176747 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:13.176747 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:13.176747 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:13.176747 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:13.176747 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:13.176747 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:13.176747 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:13.176747 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:13.176747 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:13.176747 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:13.176747 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:13.176747 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:13.177705 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:13.177705 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:13.177705 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:10:13.177705 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:10:13.177705 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:10:13.177705 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:10:13.177705 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:10:13.177705 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:10:13.177705 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:10:13.177705 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:13.181307 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 406/797] 10.222.0.50 () {56 vars in 1117 bytes} [Fri May 27 03:10:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:13.181969 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce8650-b9b0-45e1-a75c-0ad09d89eeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:13.182349 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3e9aa-24e9-4e22-be76-e0fcec655380 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:13.182772 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce8650-b9b0-45e1-a75c-0ad09d89eeda None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:13.183052 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce8650-b9b0-45e1-a75c-0ad09d89eeda None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:13.183326 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce8650-b9b0-45e1-a75c-0ad09d89eeda None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:13.193987 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ce8650-b9b0-45e1-a75c-0ad09d89eeda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:13.201572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-e7ce8650-b9b0-45e1-a75c-0ad09d89eeda None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:10:13.201572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:13.201572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:13.201572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:13.201572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:13.201572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:13.201572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:13.201572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:13.201572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:13.201572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:13.201572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:13.201572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:13.201572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:13.201572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:13.201572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:13.201572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:13.202477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:13.202477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:13.202477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:13.202477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:13.202477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:13.202477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:13.202477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:13.202477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:13.202477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:13.202477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:13.202477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:13.202477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:13.202477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:13.202477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:13.202477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:13.202477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:13.203285 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:13.203285 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:13.203285 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:13.203285 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:13.203285 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:13.203285 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:13.203285 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:13.203285 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:13.203285 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:13.203285 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:13.203285 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:10:13.203285 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:13.203285 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 392/798] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 03:10:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:13.204387 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b71391-c066-42ea-b179-a06f31b6ede4 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:13.204666 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b71391-c066-42ea-b179-a06f31b6ede4 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:13.204965 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b71391-c066-42ea-b179-a06f31b6ede4 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:13.206228 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 393/799] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:13.207104 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b71391-c066-42ea-b179-a06f31b6ede4 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:13.218471 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:13.303125 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed3e9aa-24e9-4e22-be76-e0fcec655380 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:13.317035 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b71391-c066-42ea-b179-a06f31b6ede4 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:13.319321 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:13.338099 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd3b497-0dc1-457b-90f7-032859645ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:13.338755 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd3b497-0dc1-457b-90f7-032859645ee8 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:13.339032 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd3b497-0dc1-457b-90f7-032859645ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:13.339301 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd3b497-0dc1-457b-90f7-032859645ee8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:13.360073 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1224793a-685d-4702-909f-350186f95e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:13.360746 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1224793a-685d-4702-909f-350186f95e23 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:13.361051 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1224793a-685d-4702-909f-350186f95e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:13.361323 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1224793a-685d-4702-909f-350186f95e23 None admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:13.362154 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1224793a-685d-4702-909f-350186f95e23 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:13.362453 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1224793a-685d-4702-909f-350186f95e23 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:13.364340 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1224793a-685d-4702-909f-350186f95e23 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:13.382842 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 394/800] 10.222.0.50 () {56 vars in 1149 bytes} [Fri May 27 03:10:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:13.383522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1224793a-685d-4702-909f-350186f95e23 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:13.415496 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd3b497-0dc1-457b-90f7-032859645ee8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:13.416886 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd3b497-0dc1-457b-90f7-032859645ee8 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:13.448514 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:13.498143 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1224793a-685d-4702-909f-350186f95e23 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:13.529647 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 407/801] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:10:13 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:10:13.529647 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88f382-b517-4968-9b48-3d59e57423f2 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:13.529647 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88f382-b517-4968-9b48-3d59e57423f2 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:13.529647 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88f382-b517-4968-9b48-3d59e57423f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:13.539574 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1016d0bf-ce76-4684-bae9-b6f4dc93cdfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:13.540309 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016d0bf-ce76-4684-bae9-b6f4dc93cdfb None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:13.540653 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016d0bf-ce76-4684-bae9-b6f4dc93cdfb None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:13.541006 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016d0bf-ce76-4684-bae9-b6f4dc93cdfb None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:13.541819 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1016d0bf-ce76-4684-bae9-b6f4dc93cdfb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:13.542119 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1016d0bf-ce76-4684-bae9-b6f4dc93cdfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:13.543502 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1016d0bf-ce76-4684-bae9-b6f4dc93cdfb None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:13.546178 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-ea88f382-b517-4968-9b48-3d59e57423f2 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:13.547430 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-ea88f382-b517-4968-9b48-3d59e57423f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:13.557155 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 395/802] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:10:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:13.557155 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1016d0bf-ce76-4684-bae9-b6f4dc93cdfb None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:13.582928 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:13.619754 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 408/803] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:10:13.620762 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea88f382-b517-4968-9b48-3d59e57423f2 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:13.675279 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1016d0bf-ce76-4684-bae9-b6f4dc93cdfb None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:13.689340 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:13.712174 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700a388e-68c9-437e-aceb-9cbad8e4fef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:13.712975 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a388e-68c9-437e-aceb-9cbad8e4fef3 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:13.713254 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a388e-68c9-437e-aceb-9cbad8e4fef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:13.713523 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a388e-68c9-437e-aceb-9cbad8e4fef3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:13.724502 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700a388e-68c9-437e-aceb-9cbad8e4fef3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:13.725989 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700a388e-68c9-437e-aceb-9cbad8e4fef3 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:13.736079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea88f382-b517-4968-9b48-3d59e57423f2 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:13.742832 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-700a388e-68c9-437e-aceb-9cbad8e4fef3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:13.742832 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:13.742832 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:13.742832 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:13.742832 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:13.742832 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:13.742832 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:13.742832 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:13.742832 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:13.742832 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:13.742832 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:13.742832 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:13.742832 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:13.742832 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:13.742832 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:13.743710 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:13.743710 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:13.743710 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:13.743710 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:13.743710 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:13.743710 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:13.743710 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:13.743710 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:13.743710 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:13.743710 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:13.743710 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:13.743710 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:13.743710 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:13.743710 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:13.743710 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:13.743710 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:13.743710 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:13.744591 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:13.744591 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:13.744591 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:13.744591 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:13.744591 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 396/804] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:13.744882 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700a388e-68c9-437e-aceb-9cbad8e4fef3 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:13.776352 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764b5e2-66a3-4b8c-a4af-923d69aeff75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:13.777243 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764b5e2-66a3-4b8c-a4af-923d69aeff75 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:13.777518 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764b5e2-66a3-4b8c-a4af-923d69aeff75 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:13.777786 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d764b5e2-66a3-4b8c-a4af-923d69aeff75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:13.789548 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d764b5e2-66a3-4b8c-a4af-923d69aeff75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:13.791004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d764b5e2-66a3-4b8c-a4af-923d69aeff75 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:13.811527 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-d764b5e2-66a3-4b8c-a4af-923d69aeff75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:13.811527 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:13.811527 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:13.811527 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:13.811527 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:13.811527 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:13.811527 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:13.811527 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:13.811527 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:13.811527 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:13.811527 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:13.811527 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:13.811527 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:13.811527 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:13.811527 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:13.812641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:13.812641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:13.812641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:13.812641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:13.812641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:13.812641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:13.812641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:13.812641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:13.812641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:13.812641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:13.812641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:13.812641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:13.812641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:13.812641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:13.812641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:13.812641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:13.812641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:13.813765 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:13.813765 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:13.813765 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:13.813765 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:13.815999 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 409/805] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:13.817089 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764b5e2-66a3-4b8c-a4af-923d69aeff75 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:13.831359 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:13.860137 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700a388e-68c9-437e-aceb-9cbad8e4fef3 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:13.900548 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced06517-80ae-4d83-98cc-5020e99927cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:13.901533 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced06517-80ae-4d83-98cc-5020e99927cc None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:13.901819 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced06517-80ae-4d83-98cc-5020e99927cc None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:13.902094 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced06517-80ae-4d83-98cc-5020e99927cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:13.915795 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced06517-80ae-4d83-98cc-5020e99927cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:13.917287 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced06517-80ae-4d83-98cc-5020e99927cc None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:13.930156 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d764b5e2-66a3-4b8c-a4af-923d69aeff75 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:13.934875 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-ced06517-80ae-4d83-98cc-5020e99927cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:13.934875 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:13.934875 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:13.934875 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:13.934875 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:13.934875 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:13.934875 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:13.934875 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:13.934875 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:13.934875 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:13.934875 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:13.934875 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:13.934875 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:13.934875 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:13.934875 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:13.935738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:13.935738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:13.935738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:13.935738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:13.935738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:13.935738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:13.935738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:13.935738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:13.935738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:13.935738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:13.935738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:13.935738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:13.935738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:13.935738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:13.935738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:13.935738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:13.935738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:13.936624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:13.936624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:13.936624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:13.936624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:13.940806 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 397/806] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:13.941630 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced06517-80ae-4d83-98cc-5020e99927cc None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:13.946208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:13.970050 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18049bd8-303c-4e03-85ab-e48c8af99f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:13.971012 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18049bd8-303c-4e03-85ab-e48c8af99f02 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:13.971290 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18049bd8-303c-4e03-85ab-e48c8af99f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:13.971591 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18049bd8-303c-4e03-85ab-e48c8af99f02 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:13.972278 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18049bd8-303c-4e03-85ab-e48c8af99f02 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:13.973696 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18049bd8-303c-4e03-85ab-e48c8af99f02 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:13.985721 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-18049bd8-303c-4e03-85ab-e48c8af99f02 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 03:10:13.985721 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:13.985721 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:10:13.985721 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:10:13.985721 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:10:13.985721 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:10:13.985721 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:10:13.985721 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:13.985721 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 03:10:13.985721 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:13.985721 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:10:13.985721 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:13.985721 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:13.985721 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:13.985721 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:13.985721 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:13.986676 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:13.986676 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:13.986676 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:13.986676 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:13.986676 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:13.986676 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:13.986676 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:13.986676 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:10:13.986676 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:10:13.986676 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:10:13.986676 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:10:13.986676 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:13.986676 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:13.986676 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:13.986676 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:13.986676 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:13.987535 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:13.987535 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:13.987535 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:13.987535 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:13.987535 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:13.987535 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:13.987535 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:13.987535 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:13.987535 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:13.987535 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:13.987535 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:13.987535 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:10:13.987535 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:13.987535 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 03:10:13.987535 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:13.987535 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 410/807] 10.222.0.50 () {56 vars in 1129 bytes} [Fri May 27 03:10:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:13.988693 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18049bd8-303c-4e03-85ab-e48c8af99f02 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:14.051346 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced06517-80ae-4d83-98cc-5020e99927cc None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:14.076936 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:14.086014 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f189f9f-d1f0-41ac-bd6b-ab437cea8af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:14.086672 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f189f9f-d1f0-41ac-bd6b-ab437cea8af4 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:14.086946 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f189f9f-d1f0-41ac-bd6b-ab437cea8af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:14.087214 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f189f9f-d1f0-41ac-bd6b-ab437cea8af4 None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:14.087932 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f189f9f-d1f0-41ac-bd6b-ab437cea8af4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:14.088233 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f189f9f-d1f0-41ac-bd6b-ab437cea8af4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:14.089709 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f189f9f-d1f0-41ac-bd6b-ab437cea8af4 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:14.093749 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18049bd8-303c-4e03-85ab-e48c8af99f02 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:14.106942 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 398/808] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:10:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:14.108044 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f189f9f-d1f0-41ac-bd6b-ab437cea8af4 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:14.136073 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a991ec-d264-40fe-a373-28896d4ce57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:14.136732 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a991ec-d264-40fe-a373-28896d4ce57d None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:14.137033 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a991ec-d264-40fe-a373-28896d4ce57d None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:14.137304 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a991ec-d264-40fe-a373-28896d4ce57d None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:14.138012 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a991ec-d264-40fe-a373-28896d4ce57d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:14.138309 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a991ec-d264-40fe-a373-28896d4ce57d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:14.139705 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a991ec-d264-40fe-a373-28896d4ce57d None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:14.151681 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 411/809] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:10:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:14.152457 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a991ec-d264-40fe-a373-28896d4ce57d None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:14.193081 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:14.217040 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f189f9f-d1f0-41ac-bd6b-ab437cea8af4 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:14.251040 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c02b8-8482-4980-8666-7edd2dd90433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:14.251296 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a991ec-d264-40fe-a373-28896d4ce57d None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:14.251699 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c02b8-8482-4980-8666-7edd2dd90433 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:14.251974 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c02b8-8482-4980-8666-7edd2dd90433 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:14.252237 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c02b8-8482-4980-8666-7edd2dd90433 None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:14.252998 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9c02b8-8482-4980-8666-7edd2dd90433 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:14.253539 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9c02b8-8482-4980-8666-7edd2dd90433 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:14.254950 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9c02b8-8482-4980-8666-7edd2dd90433 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:14.266559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 399/810] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:10:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:14.267798 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780a82c-dc89-4ff5-a50c-8c22d3063313 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:14.269118 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780a82c-dc89-4ff5-a50c-8c22d3063313 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:14.269425 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780a82c-dc89-4ff5-a50c-8c22d3063313 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:14.271138 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 400/811] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:14.271505 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780a82c-dc89-4ff5-a50c-8c22d3063313 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:14.294241 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f226c8d-c195-4604-a29e-65c4132e667f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:14.294981 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f226c8d-c195-4604-a29e-65c4132e667f None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:14.295270 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f226c8d-c195-4604-a29e-65c4132e667f None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:14.295574 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f226c8d-c195-4604-a29e-65c4132e667f None admin] PATH_INFO: `/v3/services` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:14.296324 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f226c8d-c195-4604-a29e-65c4132e667f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:14.296653 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f226c8d-c195-4604-a29e-65c4132e667f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:14.298648 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f226c8d-c195-4604-a29e-65c4132e667f None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:14.303660 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 412/812] 10.222.0.50 () {56 vars in 1124 bytes} [Fri May 27 03:10:14 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:14.304315 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f226c8d-c195-4604-a29e-65c4132e667f None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:14.324895 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:14.378880 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780a82c-dc89-4ff5-a50c-8c22d3063313 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:14.402005 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f226c8d-c195-4604-a29e-65c4132e667f None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:14.403541 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:14.413555 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05621ff-456d-4db3-99bc-144a6580047e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:14.414335 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05621ff-456d-4db3-99bc-144a6580047e None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:14.414609 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05621ff-456d-4db3-99bc-144a6580047e None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:14.414907 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05621ff-456d-4db3-99bc-144a6580047e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:14.421747 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05621ff-456d-4db3-99bc-144a6580047e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:14.423186 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05621ff-456d-4db3-99bc-144a6580047e None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:14.437090 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb3249-f6be-4490-9878-d08c45bdc6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:14.437724 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-e05621ff-456d-4db3-99bc-144a6580047e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:14.437724 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:14.437724 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:14.437724 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:14.437724 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:14.437724 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:14.437724 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:14.437724 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:14.437724 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:14.437724 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:14.437724 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:14.437724 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:14.437724 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:10:14.437724 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:10:14.437724 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:10:14.438751 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:10:14.438751 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:14.438751 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:14.438751 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:14.438751 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:14.438751 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:14.438751 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:14.438751 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:14.438751 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:14.438751 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:14.438751 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:14.438751 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:14.438751 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:14.438751 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:14.438751 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:14.438751 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:14.438751 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:14.439700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:10:14.439700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:10:14.439700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:10:14.439700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:10:14.439700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:10:14.439700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:10:14.439700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:14.439700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:14.440127 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb3249-f6be-4490-9878-d08c45bdc6c3 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:14.440127 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb3249-f6be-4490-9878-d08c45bdc6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:14.440127 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb3249-f6be-4490-9878-d08c45bdc6c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:14.440319 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 401/813] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:10:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:14.440319 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05621ff-456d-4db3-99bc-144a6580047e None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:14.450489 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedb3249-f6be-4490-9878-d08c45bdc6c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:14.451886 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedb3249-f6be-4490-9878-d08c45bdc6c3 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:14.466177 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-bedb3249-f6be-4490-9878-d08c45bdc6c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:14.466177 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:14.466177 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:14.466177 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:14.466177 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:14.466177 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:14.466177 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:14.466177 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:14.466177 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:14.466177 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:14.466177 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:14.466177 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:14.466177 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:10:14.466177 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:10:14.466177 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:10:14.467111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:10:14.467111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:14.467111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:14.467111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:14.467111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:14.467111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:14.467111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:14.467111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:14.467111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:14.467111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:14.467111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:14.467111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:14.467111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:14.467111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:14.467111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:14.467111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:14.467111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:14.467991 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:10:14.467991 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:10:14.467991 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:10:14.467991 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:10:14.467991 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:10:14.467991 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:10:14.467991 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:14.467991 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:14.467991 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 413/814] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:10:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:14.468626 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1bb88-9a0e-495a-8774-3505aac47e02 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:14.468841 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1bb88-9a0e-495a-8774-3505aac47e02 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:14.469117 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1bb88-9a0e-495a-8774-3505aac47e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:14.485118 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-1fe1bb88-9a0e-495a-8774-3505aac47e02 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:14.486285 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-1fe1bb88-9a0e-495a-8774-3505aac47e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:14.548432 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05621ff-456d-4db3-99bc-144a6580047e None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:14.552807 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 414/815] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:10:14.553655 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe1bb88-9a0e-495a-8774-3505aac47e02 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:14.556031 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:14.586065 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b859f-b157-42db-b49c-6bed35dd7d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:14.586866 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b859f-b157-42db-b49c-6bed35dd7d2d None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:14.587148 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b859f-b157-42db-b49c-6bed35dd7d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:14.587424 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b859f-b157-42db-b49c-6bed35dd7d2d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:14.595019 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b859f-b157-42db-b49c-6bed35dd7d2d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:14.596597 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b859f-b157-42db-b49c-6bed35dd7d2d None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:14.610967 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-e63b859f-b157-42db-b49c-6bed35dd7d2d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 03:10:14.610967 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:14.610967 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:14.610967 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:14.610967 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:14.610967 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:14.610967 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:14.610967 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:14.610967 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:14.610967 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:14.610967 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:14.610967 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:14.610967 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:10:14.610967 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:10:14.610967 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:10:14.610967 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:10:14.611972 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:14.611972 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:14.611972 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:10:14.611972 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:14.611972 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:10:14.611972 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:14.611972 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:14.611972 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:14.611972 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:14.611972 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:14.611972 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:14.611972 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:14.611972 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:14.611972 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:14.611972 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:14.611972 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:14.611972 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:14.611972 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:14.613225 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:14.613225 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:14.613225 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:10:14.613225 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:10:14.613225 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:10:14.613225 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:10:14.613225 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:10:14.613225 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:10:14.613225 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 03:10:14.613225 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:14.613225 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 402/816] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 03:10:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:14.613816 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b859f-b157-42db-b49c-6bed35dd7d2d None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:14.655309 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:14.665983 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe1bb88-9a0e-495a-8774-3505aac47e02 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:14.700784 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a89619-734c-4b02-982e-2d82d6633d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:14.701397 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a89619-734c-4b02-982e-2d82d6633d47 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:14.701788 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a89619-734c-4b02-982e-2d82d6633d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:14.702284 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a89619-734c-4b02-982e-2d82d6633d47 None admin] PATH_INFO: `/v3/services` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:14.702951 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a89619-734c-4b02-982e-2d82d6633d47 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:14.703249 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a89619-734c-4b02-982e-2d82d6633d47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:14.704625 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a89619-734c-4b02-982e-2d82d6633d47 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:14.710825 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 415/817] 10.222.0.50 () {56 vars in 1124 bytes} [Fri May 27 03:10:14 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:14.711612 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a89619-734c-4b02-982e-2d82d6633d47 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:14.723698 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b859f-b157-42db-b49c-6bed35dd7d2d None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:14.758805 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df2ac73-37f6-4ed5-a2f2-6edca28c115b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:14.759516 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2ac73-37f6-4ed5-a2f2-6edca28c115b None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:14.759790 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2ac73-37f6-4ed5-a2f2-6edca28c115b None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:14.760059 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2ac73-37f6-4ed5-a2f2-6edca28c115b None admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:14.760842 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df2ac73-37f6-4ed5-a2f2-6edca28c115b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:14.761174 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df2ac73-37f6-4ed5-a2f2-6edca28c115b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:14.762603 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df2ac73-37f6-4ed5-a2f2-6edca28c115b None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:14.774746 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 403/818] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 03:10:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:14.775544 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df2ac73-37f6-4ed5-a2f2-6edca28c115b None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:14.804632 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:14.814197 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a89619-734c-4b02-982e-2d82d6633d47 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:14.848178 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ab4a6-41d8-4084-9f52-f1b7148b1091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:14.848836 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ab4a6-41d8-4084-9f52-f1b7148b1091 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:14.849147 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ab4a6-41d8-4084-9f52-f1b7148b1091 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:14.849429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526ab4a6-41d8-4084-9f52-f1b7148b1091 None admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:14.850202 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526ab4a6-41d8-4084-9f52-f1b7148b1091 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:14.850505 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526ab4a6-41d8-4084-9f52-f1b7148b1091 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:14.851962 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526ab4a6-41d8-4084-9f52-f1b7148b1091 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:14.863713 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 416/819] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 03:10:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:14.864494 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ab4a6-41d8-4084-9f52-f1b7148b1091 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:14.877418 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df2ac73-37f6-4ed5-a2f2-6edca28c115b None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:14.887394 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:14.913635 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab007a-e83a-4895-8875-8c4323b40dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:14.914529 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab007a-e83a-4895-8875-8c4323b40dbd None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:14.914803 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab007a-e83a-4895-8875-8c4323b40dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:14.915086 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab007a-e83a-4895-8875-8c4323b40dbd None admin] PATH_INFO: `/v3/services/network` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:14.915793 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adab007a-e83a-4895-8875-8c4323b40dbd None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:14.917274 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adab007a-e83a-4895-8875-8c4323b40dbd None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:14.929477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-adab007a-e83a-4895-8875-8c4323b40dbd None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 27 03:10:14.929477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:14.929477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:10:14.929477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:10:14.929477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:10:14.929477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:10:14.929477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:10:14.929477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:14.929477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 03:10:14.929477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:14.929477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:10:14.929477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:14.929477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:14.929477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:14.929477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:14.929477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:14.929477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:14.930481 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:14.930481 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:14.930481 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:14.930481 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:14.930481 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:14.930481 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:14.930481 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:10:14.930481 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:10:14.930481 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:10:14.930481 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:10:14.930481 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:14.930481 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:14.930481 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:14.930481 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:14.930481 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:14.930481 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:14.931460 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:14.931460 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:14.931460 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:14.931460 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:14.931460 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:14.931460 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:14.931460 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:14.931460 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:14.931460 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:14.931460 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:14.931460 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:10:14.931460 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:14.931460 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 03:10:14.931460 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:14.931460 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 404/820] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:10:14 2022] GET /identity/v3/services/network => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:14.932559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab007a-e83a-4895-8875-8c4323b40dbd None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:14.976429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526ab4a6-41d8-4084-9f52-f1b7148b1091 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:15.009661 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c462b42-7548-47cb-9d93-95cdb8b40bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:15.010296 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c462b42-7548-47cb-9d93-95cdb8b40bc1 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:15.010572 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c462b42-7548-47cb-9d93-95cdb8b40bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:15.010844 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c462b42-7548-47cb-9d93-95cdb8b40bc1 None admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:15.011689 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c462b42-7548-47cb-9d93-95cdb8b40bc1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:15.011995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c462b42-7548-47cb-9d93-95cdb8b40bc1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:15.013474 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c462b42-7548-47cb-9d93-95cdb8b40bc1 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:15.018684 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:15.028291 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab007a-e83a-4895-8875-8c4323b40dbd None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:15.031658 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 417/821] 10.222.0.50 () {56 vars in 1135 bytes} [Fri May 27 03:10:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:15.032433 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c462b42-7548-47cb-9d93-95cdb8b40bc1 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:15.062393 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2440abd-232c-420c-afd5-b67c3a6a161c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:15.063051 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2440abd-232c-420c-afd5-b67c3a6a161c None admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:15.063325 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2440abd-232c-420c-afd5-b67c3a6a161c None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:15.063595 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2440abd-232c-420c-afd5-b67c3a6a161c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:15.064328 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2440abd-232c-420c-afd5-b67c3a6a161c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:15.065762 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2440abd-232c-420c-afd5-b67c3a6a161c None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:15.090078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 405/822] 10.222.0.50 () {60 vars in 1144 bytes} [Fri May 27 03:10:14 2022] POST /identity/v3/endpoints => generated 346 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:10:15.091378 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2440abd-232c-420c-afd5-b67c3a6a161c None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:15.114557 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:15.145027 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c462b42-7548-47cb-9d93-95cdb8b40bc1 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:15.193635 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2322be79-bf35-49cb-8470-ea63ad0fefad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:15.194536 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2322be79-bf35-49cb-8470-ea63ad0fefad None admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:15.194809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2322be79-bf35-49cb-8470-ea63ad0fefad None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:15.195082 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2322be79-bf35-49cb-8470-ea63ad0fefad None admin] PATH_INFO: `/v3/projects/f8398941252346468483ea0c863122f4/users/86a18e313e4944748aef58d54bfe7b13/roles/3451aa51e64642768b2228126d20fa03` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:15.211771 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2322be79-bf35-49cb-8470-ea63ad0fefad None admin] RBAC: Authorizing `identity:create_grant(project_id=f8398941252346468483ea0c863122f4, user_id=86a18e313e4944748aef58d54bfe7b13, role_id=3451aa51e64642768b2228126d20fa03)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:15.213283 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2322be79-bf35-49cb-8470-ea63ad0fefad None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:15.219121 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2440abd-232c-420c-afd5-b67c3a6a161c None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:15.231980 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 418/823] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 03:10:15 2022] PUT /identity/v3/projects/f8398941252346468483ea0c863122f4/users/86a18e313e4944748aef58d54bfe7b13/roles/3451aa51e64642768b2228126d20fa03 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:10:15.232870 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2322be79-bf35-49cb-8470-ea63ad0fefad None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:15.262047 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7bd40-d70a-4431-ad50-8fe5324a4de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:15.263090 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7bd40-d70a-4431-ad50-8fe5324a4de6 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:15.263545 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7bd40-d70a-4431-ad50-8fe5324a4de6 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:15.263960 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b7bd40-d70a-4431-ad50-8fe5324a4de6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:15.265048 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b7bd40-d70a-4431-ad50-8fe5324a4de6 None admin] RBAC: Adding query filter params (role.id=b4c524dd7bb44e94bd3b42d86e1cdf42, scope.project.id=f8398941252346468483ea0c863122f4, user.id=1a699988f08a4001b3b7d764f01f4fc5) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:15.265674 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b7bd40-d70a-4431-ad50-8fe5324a4de6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:15.267319 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b7bd40-d70a-4431-ad50-8fe5324a4de6 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:15.277044 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 406/824] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:10:15 2022] GET /identity/v3/role_assignments?user.id=1a699988f08a4001b3b7d764f01f4fc5&scope.project.id=f8398941252346468483ea0c863122f4&role.id=b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:15.277044 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7bd40-d70a-4431-ad50-8fe5324a4de6 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:15.305476 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:15.387611 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2322be79-bf35-49cb-8470-ea63ad0fefad None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:15.416283 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:15.436783 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9004e080-cf01-4a63-8219-e7b5b65e58da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:15.437568 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9004e080-cf01-4a63-8219-e7b5b65e58da None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:15.437856 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9004e080-cf01-4a63-8219-e7b5b65e58da None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:15.438131 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9004e080-cf01-4a63-8219-e7b5b65e58da None admin] PATH_INFO: `/v3/services` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:15.438574 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b7bd40-d70a-4431-ad50-8fe5324a4de6 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:15.439021 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9004e080-cf01-4a63-8219-e7b5b65e58da None admin] RBAC: Adding query filter params (name=network) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:15.439181 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9004e080-cf01-4a63-8219-e7b5b65e58da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:15.441249 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9004e080-cf01-4a63-8219-e7b5b65e58da None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:15.448541 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 419/825] 10.222.0.50 () {56 vars in 1112 bytes} [Fri May 27 03:10:15 2022] GET /identity/v3/services?name=network => generated 124 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:15.453978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9004e080-cf01-4a63-8219-e7b5b65e58da None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:15.491680 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4bdb5-2f39-49d7-bd38-a07b1d1b3ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:15.492431 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4bdb5-2f39-49d7-bd38-a07b1d1b3ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:15.493069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4bdb5-2f39-49d7-bd38-a07b1d1b3ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:15.493351 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f4bdb5-2f39-49d7-bd38-a07b1d1b3ec5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:15.509879 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f4bdb5-2f39-49d7-bd38-a07b1d1b3ec5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:15.511371 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f4bdb5-2f39-49d7-bd38-a07b1d1b3ec5 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:15.539721 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-e0f4bdb5-2f39-49d7-bd38-a07b1d1b3ec5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:15.539721 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:15.539721 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:15.539721 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:15.539721 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:15.539721 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:15.539721 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:15.539721 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:15.539721 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:15.539721 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:15.539721 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:15.539721 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:15.539721 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:15.539721 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:15.539721 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:15.540669 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:15.540669 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:15.540669 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:15.540669 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:15.540669 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:15.540669 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:15.540669 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:15.540669 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:15.540669 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:15.540669 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:15.540669 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:15.540669 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:15.540669 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:15.540669 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:15.540669 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:15.540669 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:15.540669 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:15.541616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:15.541616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:15.541616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:15.541616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:15.541845 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 407/826] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:15.553416 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4bdb5-2f39-49d7-bd38-a07b1d1b3ec5 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:15.610550 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9004e080-cf01-4a63-8219-e7b5b65e58da None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:15.632994 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:15.658135 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3cd38d-01b0-4d89-a443-0764789cff20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:15.659139 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3cd38d-01b0-4d89-a443-0764789cff20 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:15.659464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3cd38d-01b0-4d89-a443-0764789cff20 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:15.659746 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3cd38d-01b0-4d89-a443-0764789cff20 None admin] PATH_INFO: `/v3/services` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:15.660457 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3cd38d-01b0-4d89-a443-0764789cff20 None admin] RBAC: Adding query filter params (type=network) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:15.660764 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3cd38d-01b0-4d89-a443-0764789cff20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:15.663514 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3cd38d-01b0-4d89-a443-0764789cff20 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:15.669897 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 420/827] 10.222.0.50 () {56 vars in 1112 bytes} [Fri May 27 03:10:15 2022] GET /identity/v3/services?type=network => generated 350 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:15.678705 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3cd38d-01b0-4d89-a443-0764789cff20 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:15.691968 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f4bdb5-2f39-49d7-bd38-a07b1d1b3ec5 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:15.742672 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0d791d-aeb1-4fa3-bf35-bfc5db58de0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:15.743699 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d791d-aeb1-4fa3-bf35-bfc5db58de0f None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:15.744202 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d791d-aeb1-4fa3-bf35-bfc5db58de0f None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:15.744678 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d791d-aeb1-4fa3-bf35-bfc5db58de0f None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:15.746045 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0d791d-aeb1-4fa3-bf35-bfc5db58de0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:15.747038 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:15.747277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0d791d-aeb1-4fa3-bf35-bfc5db58de0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:15.750046 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0d791d-aeb1-4fa3-bf35-bfc5db58de0f None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:15.769872 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 408/828] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:10:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:15.778253 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0d791d-aeb1-4fa3-bf35-bfc5db58de0f None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:15.839355 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3cd38d-01b0-4d89-a443-0764789cff20 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:15.920698 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0d791d-aeb1-4fa3-bf35-bfc5db58de0f None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:15.935936 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c8d68-cf89-49b8-a7d2-4c6d9d40e137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:15.936605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c8d68-cf89-49b8-a7d2-4c6d9d40e137 None admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:15.936883 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c8d68-cf89-49b8-a7d2-4c6d9d40e137 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:15.937185 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4c8d68-cf89-49b8-a7d2-4c6d9d40e137 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:15.937936 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4c8d68-cf89-49b8-a7d2-4c6d9d40e137 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:15.939311 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4c8d68-cf89-49b8-a7d2-4c6d9d40e137 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:15.961577 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:15.974080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48cf963-4702-4171-b9d6-cac6494da269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:15.974933 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48cf963-4702-4171-b9d6-cac6494da269 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:15.975216 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48cf963-4702-4171-b9d6-cac6494da269 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:15.975492 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48cf963-4702-4171-b9d6-cac6494da269 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:15.984522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48cf963-4702-4171-b9d6-cac6494da269 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:15.986371 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48cf963-4702-4171-b9d6-cac6494da269 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:15.997109 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 421/829] 10.222.0.50 () {60 vars in 1144 bytes} [Fri May 27 03:10:15 2022] POST /identity/v3/endpoints => generated 327 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:10:16.011178 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-e48cf963-4702-4171-b9d6-cac6494da269 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:16.011178 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:16.011178 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:16.011178 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:16.011178 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:16.011178 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:16.011178 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:16.011178 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:16.011178 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:16.011178 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:16.011178 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:16.011178 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:16.011178 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:10:16.011178 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:10:16.011178 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:10:16.012376 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:10:16.012376 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:16.012376 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:16.012376 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:16.012376 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:16.012376 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:16.012376 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:16.012376 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:16.012376 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:16.012376 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:16.012376 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:16.012376 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:16.012376 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:16.012376 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:16.012376 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:16.012376 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:16.012376 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:16.013372 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:10:16.013372 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:10:16.013372 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:10:16.013372 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:10:16.013372 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:10:16.013372 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:10:16.013372 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:16.013372 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:16.016833 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 409/830] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:10:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:16.019950 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c8d68-cf89-49b8-a7d2-4c6d9d40e137 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:16.147138 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:16.151693 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4c8d68-cf89-49b8-a7d2-4c6d9d40e137 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:16.191112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c1a466-293d-48f4-a6e8-73b0b06507a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:16.191781 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c1a466-293d-48f4-a6e8-73b0b06507a3 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:16.192055 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c1a466-293d-48f4-a6e8-73b0b06507a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:16.192330 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c1a466-293d-48f4-a6e8-73b0b06507a3 None admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:16.193147 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c1a466-293d-48f4-a6e8-73b0b06507a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:16.193526 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c1a466-293d-48f4-a6e8-73b0b06507a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:16.194983 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c1a466-293d-48f4-a6e8-73b0b06507a3 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:16.208815 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 422/831] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 03:10:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:16.214839 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48cf963-4702-4171-b9d6-cac6494da269 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:16.330198 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48cf963-4702-4171-b9d6-cac6494da269 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:16.369445 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f2e3b-2857-4e47-821d-656c6ed0fdd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:16.370357 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f2e3b-2857-4e47-821d-656c6ed0fdd6 None admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:16.370633 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f2e3b-2857-4e47-821d-656c6ed0fdd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:16.370906 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f2e3b-2857-4e47-821d-656c6ed0fdd6 None admin] PATH_INFO: `/v3/projects/f8398941252346468483ea0c863122f4/users/3de53e2f0fba4e0fac70887067c1963d/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:16.372932 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098f2e3b-2857-4e47-821d-656c6ed0fdd6 None admin] RBAC: Authorizing `identity:create_grant(project_id=f8398941252346468483ea0c863122f4, user_id=3de53e2f0fba4e0fac70887067c1963d, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:16.375596 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098f2e3b-2857-4e47-821d-656c6ed0fdd6 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:16.390779 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 410/832] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 03:10:16 2022] PUT /identity/v3/projects/f8398941252346468483ea0c863122f4/users/3de53e2f0fba4e0fac70887067c1963d/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:10:17.682267 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d2103-a8c7-4539-ae8e-6dc0d97ddd23 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:17.682784 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d2103-a8c7-4539-ae8e-6dc0d97ddd23 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:17.682863 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d2103-a8c7-4539-ae8e-6dc0d97ddd23 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:17.684430 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 423/833] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:17.689634 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47adc0ff-aa95-4bad-b2ec-dfa015cd4776 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:17.690080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47adc0ff-aa95-4bad-b2ec-dfa015cd4776 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:17.690168 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47adc0ff-aa95-4bad-b2ec-dfa015cd4776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:17.704842 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b1662-5a86-46d8-90f4-b49cc974ade9 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:17.705185 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b1662-5a86-46d8-90f4-b49cc974ade9 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:17.705426 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b1662-5a86-46d8-90f4-b49cc974ade9 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:17.707178 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 424/834] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:10:17.708063 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-47adc0ff-aa95-4bad-b2ec-dfa015cd4776 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:17.709549 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-47adc0ff-aa95-4bad-b2ec-dfa015cd4776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:17.712302 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452500b-e2c2-4413-8b81-5f1f6577b372 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:17.712591 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452500b-e2c2-4413-8b81-5f1f6577b372 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:17.712872 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452500b-e2c2-4413-8b81-5f1f6577b372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:17.729922 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-c452500b-e2c2-4413-8b81-5f1f6577b372 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:17.731110 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-c452500b-e2c2-4413-8b81-5f1f6577b372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:17.751989 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:17.795324 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 411/835] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:10:17.819265 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47adc0ff-aa95-4bad-b2ec-dfa015cd4776 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:17.827758 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 425/836] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:10:17.850404 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c452500b-e2c2-4413-8b81-5f1f6577b372 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:17.926628 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47adc0ff-aa95-4bad-b2ec-dfa015cd4776 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:17.943395 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:17.953622 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c452500b-e2c2-4413-8b81-5f1f6577b372 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:17.966274 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb44861-6142-439d-a6bb-d67ba1ab54d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:17.967520 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb44861-6142-439d-a6bb-d67ba1ab54d7 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:17.968011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb44861-6142-439d-a6bb-d67ba1ab54d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:17.968490 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb44861-6142-439d-a6bb-d67ba1ab54d7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:17.980818 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb44861-6142-439d-a6bb-d67ba1ab54d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:17.989615 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-2cb44861-6142-439d-a6bb-d67ba1ab54d7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:10:17.989615 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:17.989615 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:17.989615 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:17.989615 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:17.989615 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:17.989615 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:17.989615 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:17.989615 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:17.989615 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:17.989615 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:17.989615 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:17.989615 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:17.989615 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:17.989615 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:17.989615 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:17.990659 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:17.990659 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:17.990659 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:17.990659 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:17.990659 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:17.990659 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:17.990659 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:17.990659 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:17.990659 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:17.990659 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:17.990659 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:17.990659 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:17.990659 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:17.990659 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:17.990659 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:17.990659 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:17.991573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:17.991573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:17.991573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:17.991573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:17.991573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:17.991573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:17.991573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:17.991573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:17.991573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:17.991573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:17.991573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:10:17.991573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:17.992282 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4100a0dc-83ab-4a34-bd54-588221248830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:17.993208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100a0dc-83ab-4a34-bd54-588221248830 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:17.993716 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100a0dc-83ab-4a34-bd54-588221248830 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:17.994145 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100a0dc-83ab-4a34-bd54-588221248830 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:17.995214 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4100a0dc-83ab-4a34-bd54-588221248830 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:17.999902 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4100a0dc-83ab-4a34-bd54-588221248830 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:18.000453 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 412/837] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 03:10:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:10:18.001649 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb44861-6142-439d-a6bb-d67ba1ab54d7 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:18.017709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-4100a0dc-83ab-4a34-bd54-588221248830 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 03:10:18.017709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:18.017709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:10:18.017709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:10:18.017709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:10:18.017709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:10:18.017709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:10:18.017709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:18.017709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 03:10:18.017709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:18.017709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:10:18.017709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:18.017709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:18.017709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:18.017709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:18.017709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:18.017709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:18.019043 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:18.019043 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:18.019043 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:18.019043 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:18.019043 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:18.019043 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:18.019043 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:10:18.019043 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:10:18.019043 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:10:18.019043 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:10:18.019043 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:18.019043 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:18.019043 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:18.019043 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:18.019043 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:18.019043 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:18.020082 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:18.020082 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:18.020082 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:18.020082 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:18.020082 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:18.020082 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:18.020082 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:18.020082 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:18.020082 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:18.020082 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:18.020082 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:10:18.020082 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:18.020082 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 03:10:18.020082 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:18.025123 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 426/838] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 03:10:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:10:18.027160 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551beb55-3454-4fab-95d3-e3b6440f06f2 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:18.027438 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551beb55-3454-4fab-95d3-e3b6440f06f2 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:18.027712 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551beb55-3454-4fab-95d3-e3b6440f06f2 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:18.029672 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 427/839] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:18.034727 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551beb55-3454-4fab-95d3-e3b6440f06f2 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:18.039545 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:18.115670 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb44861-6142-439d-a6bb-d67ba1ab54d7 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:18.152013 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551beb55-3454-4fab-95d3-e3b6440f06f2 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:18.152221 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe62b0f-698a-441b-a668-75f5351cb38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:18.152804 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe62b0f-698a-441b-a668-75f5351cb38c None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:18.153119 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe62b0f-698a-441b-a668-75f5351cb38c None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:18.153385 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe62b0f-698a-441b-a668-75f5351cb38c None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:18.154153 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe62b0f-698a-441b-a668-75f5351cb38c None admin] RBAC: Adding query filter params (name=service) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:18.154448 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe62b0f-698a-441b-a668-75f5351cb38c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:18.155853 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe62b0f-698a-441b-a668-75f5351cb38c None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:18.169478 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 413/840] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 03:10:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:18.170833 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c4a5b-a88e-49e3-9e44-ee730a0c8102 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:18.171121 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c4a5b-a88e-49e3-9e44-ee730a0c8102 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:18.171399 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c4a5b-a88e-49e3-9e44-ee730a0c8102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:18.189295 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-348c4a5b-a88e-49e3-9e44-ee730a0c8102 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:18.190474 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-348c4a5b-a88e-49e3-9e44-ee730a0c8102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:18.190814 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68c6ac-6492-4a4c-ada1-85f17c9bd180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:18.191502 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68c6ac-6492-4a4c-ada1-85f17c9bd180 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:18.191785 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68c6ac-6492-4a4c-ada1-85f17c9bd180 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:18.192059 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68c6ac-6492-4a4c-ada1-85f17c9bd180 None admin] PATH_INFO: `/v3/services` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:18.192968 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d68c6ac-6492-4a4c-ada1-85f17c9bd180 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:18.193298 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d68c6ac-6492-4a4c-ada1-85f17c9bd180 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:18.195579 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d68c6ac-6492-4a4c-ada1-85f17c9bd180 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:18.205624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 428/841] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 03:10:18 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:18.205624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68c6ac-6492-4a4c-ada1-85f17c9bd180 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:18.225390 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:18.265936 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 414/842] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:18 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:10:18.270182 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c4a5b-a88e-49e3-9e44-ee730a0c8102 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:18.298670 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:18.322220 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68c6ac-6492-4a4c-ada1-85f17c9bd180 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:18.359411 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0265bdeb-276f-48bd-ac39-74b92a7c7d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:18.360240 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265bdeb-276f-48bd-ac39-74b92a7c7d5d None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:18.360508 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265bdeb-276f-48bd-ac39-74b92a7c7d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:18.360774 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0265bdeb-276f-48bd-ac39-74b92a7c7d5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:18.374708 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0265bdeb-276f-48bd-ac39-74b92a7c7d5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:18.376306 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0265bdeb-276f-48bd-ac39-74b92a7c7d5d None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:18.379354 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c4a5b-a88e-49e3-9e44-ee730a0c8102 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:18.397834 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-0265bdeb-276f-48bd-ac39-74b92a7c7d5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:18.397834 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:18.397834 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:18.397834 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:18.397834 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:18.397834 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:18.397834 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:18.397834 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:18.397834 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:18.397834 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:18.397834 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:18.397834 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:18.397834 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:18.397834 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:18.397834 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:18.399040 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:18.399040 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:18.399040 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:18.399040 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:18.399040 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:18.399040 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:18.399040 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:18.399040 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:18.399040 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:18.399040 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:18.399040 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:18.399040 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:18.399040 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:18.399040 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:18.399040 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:18.399040 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:18.399040 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:18.405056 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:18.405056 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:18.405056 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:18.405056 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:18.405056 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 429/843] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:18.405056 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0265bdeb-276f-48bd-ac39-74b92a7c7d5d None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:18.421301 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62a15e2-fed5-4326-83bc-ec97a833e994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:18.422301 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a15e2-fed5-4326-83bc-ec97a833e994 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:18.422653 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a15e2-fed5-4326-83bc-ec97a833e994 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:18.422924 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a15e2-fed5-4326-83bc-ec97a833e994 None admin] PATH_INFO: `/v3/services` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:18.423639 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62a15e2-fed5-4326-83bc-ec97a833e994 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:18.423938 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62a15e2-fed5-4326-83bc-ec97a833e994 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:18.425862 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62a15e2-fed5-4326-83bc-ec97a833e994 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:18.437628 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 415/844] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 03:10:18 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:18.437628 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62a15e2-fed5-4326-83bc-ec97a833e994 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:18.513126 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:18.553269 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:18.560662 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0265bdeb-276f-48bd-ac39-74b92a7c7d5d None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:18.569135 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62a15e2-fed5-4326-83bc-ec97a833e994 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:18.600643 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb8aeb6-52b8-4a06-b15b-6f0f41623f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:18.601935 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb8aeb6-52b8-4a06-b15b-6f0f41623f75 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:18.602437 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb8aeb6-52b8-4a06-b15b-6f0f41623f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:18.602912 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb8aeb6-52b8-4a06-b15b-6f0f41623f75 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:18.603797 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb8aeb6-52b8-4a06-b15b-6f0f41623f75 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:18.605546 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb8aeb6-52b8-4a06-b15b-6f0f41623f75 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:18.607481 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad5569-9f19-4483-ba71-147e7464b95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:18.608292 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad5569-9f19-4483-ba71-147e7464b95d None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:18.608474 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad5569-9f19-4483-ba71-147e7464b95d None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:18.608782 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad5569-9f19-4483-ba71-147e7464b95d None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:18.609651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad5569-9f19-4483-ba71-147e7464b95d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:18.609965 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad5569-9f19-4483-ba71-147e7464b95d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:18.611418 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad5569-9f19-4483-ba71-147e7464b95d None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:18.621367 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-bcb8aeb6-52b8-4a06-b15b-6f0f41623f75 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 03:10:18.621367 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:18.621367 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:10:18.621367 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:10:18.621367 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:10:18.621367 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:10:18.621367 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:10:18.621367 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:18.621367 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 03:10:18.621367 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:18.621367 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:10:18.621367 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:18.621367 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:18.621367 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:18.621367 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:18.621367 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:18.622380 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:18.622380 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:18.622380 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:18.622380 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:18.622380 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:18.622380 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:18.622380 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:18.622380 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:10:18.622380 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:10:18.622380 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:10:18.622380 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:10:18.622380 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:18.622380 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:18.622380 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:18.622380 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:18.622380 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:18.623182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:18.623182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:18.623182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:18.623182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:18.623182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:18.623182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:18.623182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:18.623182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:18.623182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:18.623182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:18.623182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:18.623182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:10:18.623182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:18.623182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 03:10:18.623182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:18.626135 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 430/845] 10.222.0.50 () {56 vars in 1132 bytes} [Fri May 27 03:10:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:18.636004 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 416/846] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:10:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:18.636215 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb8aeb6-52b8-4a06-b15b-6f0f41623f75 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:18.636999 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad5569-9f19-4483-ba71-147e7464b95d None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:18.753384 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb8aeb6-52b8-4a06-b15b-6f0f41623f75 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:18.764472 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad5569-9f19-4483-ba71-147e7464b95d None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:18.792218 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f5c41-5aa8-4e4d-8afa-ab022b8e8f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:18.793170 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f5c41-5aa8-4e4d-8afa-ab022b8e8f5d None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:18.793448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f5c41-5aa8-4e4d-8afa-ab022b8e8f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:18.793717 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f5c41-5aa8-4e4d-8afa-ab022b8e8f5d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:18.795962 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:18.804279 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb7e44-6737-407b-ab77-00c12a701785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:18.804940 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26f5c41-5aa8-4e4d-8afa-ab022b8e8f5d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:18.805367 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb7e44-6737-407b-ab77-00c12a701785 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:18.805819 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb7e44-6737-407b-ab77-00c12a701785 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:18.806095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb7e44-6737-407b-ab77-00c12a701785 None admin] PATH_INFO: `/v3/services` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:18.806803 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebb7e44-6737-407b-ab77-00c12a701785 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:18.806914 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26f5c41-5aa8-4e4d-8afa-ab022b8e8f5d None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:18.807110 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebb7e44-6737-407b-ab77-00c12a701785 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:18.808469 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebb7e44-6737-407b-ab77-00c12a701785 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:18.818129 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 417/847] 10.222.0.50 () {56 vars in 1126 bytes} [Fri May 27 03:10:18 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:18.818129 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff7f01-cb26-4f7f-96ca-f6a3ce0c69dc None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:18.818129 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff7f01-cb26-4f7f-96ca-f6a3ce0c69dc None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:18.818129 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff7f01-cb26-4f7f-96ca-f6a3ce0c69dc None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:18.818129 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 418/848] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:18.821694 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daff7f01-cb26-4f7f-96ca-f6a3ce0c69dc None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:18.828293 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-d26f5c41-5aa8-4e4d-8afa-ab022b8e8f5d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:10:18.828293 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:18.828293 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:18.828293 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:18.828293 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:18.828293 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:18.828293 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:18.828293 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:18.828293 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:18.828293 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:18.828293 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:18.828293 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:18.828293 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:10:18.828293 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:10:18.828293 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:10:18.828293 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:10:18.829464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:18.829464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:18.829464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:10:18.829464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:18.829464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:10:18.829464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:18.829464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:18.829464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:18.829464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:18.829464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:18.829464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:18.829464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:18.829464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:18.829464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:18.829464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:18.829464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:18.829464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:18.829464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:18.830396 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:18.830396 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:18.830396 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:10:18.830396 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:10:18.830396 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:10:18.830396 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:10:18.830396 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:10:18.830396 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:10:18.830396 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:10:18.830396 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:18.833588 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 431/849] 10.222.0.50 () {56 vars in 1117 bytes} [Fri May 27 03:10:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:18.835250 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bdb024-6cfd-4cc9-97e8-36462582dc52 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:18.835528 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bdb024-6cfd-4cc9-97e8-36462582dc52 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:18.835797 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bdb024-6cfd-4cc9-97e8-36462582dc52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:18.841808 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:18.853529 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-e4bdb024-6cfd-4cc9-97e8-36462582dc52 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:18.854719 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-e4bdb024-6cfd-4cc9-97e8-36462582dc52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:18.931023 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 432/850] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:18 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:10:18.931023 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bdb024-6cfd-4cc9-97e8-36462582dc52 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:18.943568 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daff7f01-cb26-4f7f-96ca-f6a3ce0c69dc None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:18.979524 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ae0d0-df2c-4393-9a20-06b8e30c6f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:18.980543 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ae0d0-df2c-4393-9a20-06b8e30c6f03 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:18.980821 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ae0d0-df2c-4393-9a20-06b8e30c6f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:18.981141 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ae0d0-df2c-4393-9a20-06b8e30c6f03 None admin] PATH_INFO: `/v3/services` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:18.981853 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2ae0d0-df2c-4393-9a20-06b8e30c6f03 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:18.982156 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2ae0d0-df2c-4393-9a20-06b8e30c6f03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:18.983604 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2ae0d0-df2c-4393-9a20-06b8e30c6f03 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:18.988273 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 419/851] 10.222.0.50 () {56 vars in 1126 bytes} [Fri May 27 03:10:18 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:18.997241 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ae0d0-df2c-4393-9a20-06b8e30c6f03 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:19.065298 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bdb024-6cfd-4cc9-97e8-36462582dc52 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:19.084413 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:19.107320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:19.116399 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ae0d0-df2c-4393-9a20-06b8e30c6f03 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:19.123973 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901adb3e-e59a-4635-b009-1e8821844347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:19.125796 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901adb3e-e59a-4635-b009-1e8821844347 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:19.126295 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901adb3e-e59a-4635-b009-1e8821844347 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:19.127257 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901adb3e-e59a-4635-b009-1e8821844347 None admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:19.128800 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901adb3e-e59a-4635-b009-1e8821844347 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:19.129410 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901adb3e-e59a-4635-b009-1e8821844347 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:19.131673 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901adb3e-e59a-4635-b009-1e8821844347 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:19.156710 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc7ff66-ec9f-4fd6-b42a-3d3174df247a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:19.157715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7ff66-ec9f-4fd6-b42a-3d3174df247a None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:19.158064 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7ff66-ec9f-4fd6-b42a-3d3174df247a None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:19.158594 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 433/852] 10.222.0.50 () {56 vars in 1149 bytes} [Fri May 27 03:10:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:19.159435 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7ff66-ec9f-4fd6-b42a-3d3174df247a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:19.165298 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901adb3e-e59a-4635-b009-1e8821844347 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:19.176057 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc7ff66-ec9f-4fd6-b42a-3d3174df247a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:19.184142 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-bdc7ff66-ec9f-4fd6-b42a-3d3174df247a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:10:19.184142 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:19.184142 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:19.184142 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:19.184142 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:19.184142 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:19.184142 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:19.184142 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:19.184142 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:19.184142 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:19.184142 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:19.184142 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:19.184142 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:19.184142 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:19.184142 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:19.184142 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:19.185182 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:19.185182 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:19.185182 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:19.185182 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:19.185182 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:19.185182 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:19.185182 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:19.185182 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:19.185182 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:19.185182 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:19.185182 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:19.185182 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:19.185182 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:19.185182 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:19.185182 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:19.185182 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:19.186021 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:19.186021 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:19.186021 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:19.186021 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:19.186021 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:19.186021 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:19.186021 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:19.186021 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:19.186021 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:19.186021 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:19.186021 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:10:19.186021 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:19.186021 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 420/853] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 03:10:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:19.191561 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc7ff66-ec9f-4fd6-b42a-3d3174df247a None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:19.288422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901adb3e-e59a-4635-b009-1e8821844347 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:19.325806 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae37c90c-64ed-4f48-bcec-19477926c4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:19.326980 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae37c90c-64ed-4f48-bcec-19477926c4a2 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:19.327780 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae37c90c-64ed-4f48-bcec-19477926c4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:19.328531 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae37c90c-64ed-4f48-bcec-19477926c4a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:19.330215 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc7ff66-ec9f-4fd6-b42a-3d3174df247a None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:19.370901 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee48efcf-09d8-4fa9-a3b8-0d64fd16822c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:19.371580 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48efcf-09d8-4fa9-a3b8-0d64fd16822c None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:19.371862 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48efcf-09d8-4fa9-a3b8-0d64fd16822c None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:19.372142 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee48efcf-09d8-4fa9-a3b8-0d64fd16822c None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:19.372897 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee48efcf-09d8-4fa9-a3b8-0d64fd16822c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:19.373234 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee48efcf-09d8-4fa9-a3b8-0d64fd16822c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:19.374616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee48efcf-09d8-4fa9-a3b8-0d64fd16822c None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:19.381287 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:19.389050 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 421/854] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:10:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:19.391578 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee48efcf-09d8-4fa9-a3b8-0d64fd16822c None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:19.394053 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:19.422804 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae37c90c-64ed-4f48-bcec-19477926c4a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:19.425515 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae37c90c-64ed-4f48-bcec-19477926c4a2 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:19.527837 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee48efcf-09d8-4fa9-a3b8-0d64fd16822c None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:19.558501 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 434/855] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:10:19 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:10:19.563549 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae37c90c-64ed-4f48-bcec-19477926c4a2 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:19.572982 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac98568-10a5-4b15-9f2b-01e6ed31f339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:19.573791 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac98568-10a5-4b15-9f2b-01e6ed31f339 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:19.574068 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac98568-10a5-4b15-9f2b-01e6ed31f339 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:19.574335 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac98568-10a5-4b15-9f2b-01e6ed31f339 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:19.587702 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac98568-10a5-4b15-9f2b-01e6ed31f339 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:19.589338 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac98568-10a5-4b15-9f2b-01e6ed31f339 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:19.609003 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-bac98568-10a5-4b15-9f2b-01e6ed31f339 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:19.609003 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:19.609003 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:19.609003 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:19.609003 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:19.609003 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:19.609003 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:19.609003 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:19.609003 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:19.609003 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:19.609003 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:19.609003 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:19.609003 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:19.609003 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:19.609003 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:19.610011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:19.610011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:19.610011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:19.610011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:19.610011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:19.610011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:19.610011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:19.610011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:19.610011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:19.610011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:19.610011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:19.610011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:19.610011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:19.610011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:19.610011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:19.610011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:19.610011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:19.610931 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:19.610931 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:19.610931 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:19.610931 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:19.610931 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 422/856] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:10:19.616204 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac98568-10a5-4b15-9f2b-01e6ed31f339 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:19.669163 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:19.678245 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:19.688345 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae37c90c-64ed-4f48-bcec-19477926c4a2 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:19.724279 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac98568-10a5-4b15-9f2b-01e6ed31f339 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:19.726161 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286184ad-ef08-4fbf-b89f-072cf7a1d8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:19.727230 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286184ad-ef08-4fbf-b89f-072cf7a1d8d3 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:19.727745 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286184ad-ef08-4fbf-b89f-072cf7a1d8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:19.728213 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286184ad-ef08-4fbf-b89f-072cf7a1d8d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:19.739719 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286184ad-ef08-4fbf-b89f-072cf7a1d8d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:19.741553 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286184ad-ef08-4fbf-b89f-072cf7a1d8d3 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:19.760173 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-286184ad-ef08-4fbf-b89f-072cf7a1d8d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:19.760173 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:19.760173 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:19.760173 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:19.760173 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:19.760173 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:19.760173 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:19.760173 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:19.760173 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:19.760173 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:19.760173 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:19.760173 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:19.760173 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:19.760173 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:19.760173 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:19.761609 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:19.761609 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:19.761609 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:19.761609 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:19.761609 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:19.761609 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:19.761609 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:19.761609 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:19.761609 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:19.761609 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:19.761609 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:19.761609 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:19.761609 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:19.761609 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:19.761609 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:19.761609 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:19.761609 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:19.762530 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:19.762530 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:19.762530 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:19.762530 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:19.762751 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8336d6ae-0c8d-47e9-863f-49417b5a223d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:19.762751 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336d6ae-0c8d-47e9-863f-49417b5a223d None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:19.762751 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336d6ae-0c8d-47e9-863f-49417b5a223d None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:19.762751 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336d6ae-0c8d-47e9-863f-49417b5a223d None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:19.763034 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8336d6ae-0c8d-47e9-863f-49417b5a223d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:19.763034 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8336d6ae-0c8d-47e9-863f-49417b5a223d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:19.763888 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8336d6ae-0c8d-47e9-863f-49417b5a223d None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:19.765504 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 435/857] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:10:19.771321 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286184ad-ef08-4fbf-b89f-072cf7a1d8d3 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:19.781848 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 423/858] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:10:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:19.783172 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8336d6ae-0c8d-47e9-863f-49417b5a223d None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:19.893127 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286184ad-ef08-4fbf-b89f-072cf7a1d8d3 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:19.912534 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8336d6ae-0c8d-47e9-863f-49417b5a223d None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:19.927291 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:19.931956 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad7b83-60d8-4a2a-a62e-c94ccdd30a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:19.933116 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad7b83-60d8-4a2a-a62e-c94ccdd30a32 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:19.933471 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad7b83-60d8-4a2a-a62e-c94ccdd30a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:19.933815 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad7b83-60d8-4a2a-a62e-c94ccdd30a32 None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:19.934876 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dad7b83-60d8-4a2a-a62e-c94ccdd30a32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:19.935309 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dad7b83-60d8-4a2a-a62e-c94ccdd30a32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:19.937218 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dad7b83-60d8-4a2a-a62e-c94ccdd30a32 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:19.938881 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:19.956131 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec7177-bd47-4414-84c8-8b298b367f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:19.957422 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec7177-bd47-4414-84c8-8b298b367f9b None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:19.957897 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec7177-bd47-4414-84c8-8b298b367f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:19.958301 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec7177-bd47-4414-84c8-8b298b367f9b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:19.958872 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 436/859] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:10:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:19.959828 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad7b83-60d8-4a2a-a62e-c94ccdd30a32 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:19.967647 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ec7177-bd47-4414-84c8-8b298b367f9b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:19.969395 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ec7177-bd47-4414-84c8-8b298b367f9b None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:19.986180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-64ec7177-bd47-4414-84c8-8b298b367f9b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 03:10:19.986180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:19.986180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:19.986180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:19.986180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:19.986180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:19.986180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:19.986180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:19.986180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:19.986180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:19.986180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:19.986180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:19.986180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:10:19.986180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:10:19.986180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:10:19.986180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:10:19.987410 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:19.987410 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:19.987410 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:10:19.987410 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:19.987410 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:10:19.987410 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:19.987410 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:19.987410 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:19.987410 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:19.987410 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:19.987410 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:19.987410 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:19.987410 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:19.987410 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:19.987410 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:19.987410 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:19.987410 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:19.987410 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:19.988666 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:19.988666 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:19.988666 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:10:19.988666 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:10:19.988666 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:10:19.988666 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:10:19.988666 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:10:19.988666 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:10:19.988666 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 03:10:19.988666 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:19.990841 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 424/860] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 03:10:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:19.996393 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec7177-bd47-4414-84c8-8b298b367f9b None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:20.095323 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad7b83-60d8-4a2a-a62e-c94ccdd30a32 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:20.104927 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec7177-bd47-4414-84c8-8b298b367f9b None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:20.139524 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f716a-8805-4cd7-8ab8-608f28e93ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:20.140645 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f716a-8805-4cd7-8ab8-608f28e93ece None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:20.141047 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f716a-8805-4cd7-8ab8-608f28e93ece None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:20.141423 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f716a-8805-4cd7-8ab8-608f28e93ece None admin] PATH_INFO: `/v3/projects/service` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:20.143887 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fabb57-bbd6-4e9e-879e-f913cb27f0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:20.144981 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fabb57-bbd6-4e9e-879e-f913cb27f0cf None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:20.145389 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fabb57-bbd6-4e9e-879e-f913cb27f0cf None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:20.145815 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fabb57-bbd6-4e9e-879e-f913cb27f0cf None admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:20.146843 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fabb57-bbd6-4e9e-879e-f913cb27f0cf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:20.147327 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fabb57-bbd6-4e9e-879e-f913cb27f0cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:20.148884 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fabb57-bbd6-4e9e-879e-f913cb27f0cf None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:20.150489 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969f716a-8805-4cd7-8ab8-608f28e93ece None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:20.153267 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969f716a-8805-4cd7-8ab8-608f28e93ece None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:20.171511 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 425/861] 10.222.0.50 () {56 vars in 1135 bytes} [Fri May 27 03:10:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:20.176471 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-969f716a-8805-4cd7-8ab8-608f28e93ece None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:20.176471 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:20.176471 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:20.176471 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:20.176471 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:20.176471 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:20.176471 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:20.176471 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:20.176471 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:20.176471 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:20.176471 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:20.176471 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:20.176471 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:10:20.176471 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:10:20.176471 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:10:20.177543 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:10:20.177543 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:20.177543 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:20.177543 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:20.177543 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:20.177543 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:20.177543 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:20.177543 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:20.177543 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:20.177543 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:20.177543 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:20.177543 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:20.177543 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:20.177543 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:20.177543 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:20.177543 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:20.177543 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:20.178638 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:10:20.178638 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:10:20.178638 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:10:20.178638 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:10:20.178638 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:10:20.178638 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:10:20.178638 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:20.178638 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:20.179837 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fabb57-bbd6-4e9e-879e-f913cb27f0cf None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:20.180861 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 437/862] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:10:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:20.186113 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f716a-8805-4cd7-8ab8-608f28e93ece None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:20.214732 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:20.248727 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:20.303566 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f716a-8805-4cd7-8ab8-608f28e93ece None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:20.316527 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fabb57-bbd6-4e9e-879e-f913cb27f0cf None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:20.341731 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3704bf86-4f55-4fca-ae10-3798f2ba0d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:20.342644 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704bf86-4f55-4fca-ae10-3798f2ba0d02 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:20.343087 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704bf86-4f55-4fca-ae10-3798f2ba0d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:20.343492 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704bf86-4f55-4fca-ae10-3798f2ba0d02 None admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:20.344564 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3704bf86-4f55-4fca-ae10-3798f2ba0d02 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:20.345129 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3704bf86-4f55-4fca-ae10-3798f2ba0d02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:20.346900 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3704bf86-4f55-4fca-ae10-3798f2ba0d02 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:20.353072 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadeb338-dcbb-47cb-bc7c-7437fd93e7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:20.353878 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadeb338-dcbb-47cb-bc7c-7437fd93e7da None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:20.354258 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadeb338-dcbb-47cb-bc7c-7437fd93e7da None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:20.354632 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadeb338-dcbb-47cb-bc7c-7437fd93e7da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:20.361529 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 438/863] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 03:10:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:20.369059 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3704bf86-4f55-4fca-ae10-3798f2ba0d02 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:20.443049 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadeb338-dcbb-47cb-bc7c-7437fd93e7da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:20.445499 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadeb338-dcbb-47cb-bc7c-7437fd93e7da None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:20.474506 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:20.487489 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3704bf86-4f55-4fca-ae10-3798f2ba0d02 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:20.506386 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:20.537084 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ccf97-440f-469f-b9cb-4cc4c29826c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:20.538224 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ccf97-440f-469f-b9cb-4cc4c29826c4 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:20.538606 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ccf97-440f-469f-b9cb-4cc4c29826c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:20.538982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ccf97-440f-469f-b9cb-4cc4c29826c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:20.540211 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91ccf97-440f-469f-b9cb-4cc4c29826c4 None admin] RBAC: Adding query filter params (role.id=3451aa51e64642768b2228126d20fa03, scope.project.id=f8398941252346468483ea0c863122f4, user.id=86a18e313e4944748aef58d54bfe7b13) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:20.540636 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91ccf97-440f-469f-b9cb-4cc4c29826c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:20.543185 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91ccf97-440f-469f-b9cb-4cc4c29826c4 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:20.551043 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 439/864] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:10:20 2022] GET /identity/v3/role_assignments?user.id=86a18e313e4944748aef58d54bfe7b13&scope.project.id=f8398941252346468483ea0c863122f4&role.id=3451aa51e64642768b2228126d20fa03 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:20.570616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 426/865] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:10:20 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:10:20.576257 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ccf97-440f-469f-b9cb-4cc4c29826c4 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:20.687014 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ccf97-440f-469f-b9cb-4cc4c29826c4 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:20.716792 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218383f-193e-4fb9-a410-f4028f296ae9 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:20.717032 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218383f-193e-4fb9-a410-f4028f296ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:20.717321 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b218383f-193e-4fb9-a410-f4028f296ae9 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:20.718977 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 427/866] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:10:20.725054 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6853540-22c3-439c-9d1b-1bf0aa70bf84 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:20.725347 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6853540-22c3-439c-9d1b-1bf0aa70bf84 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:20.725616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6853540-22c3-439c-9d1b-1bf0aa70bf84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:20.726183 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8464e6db-1130-4cf2-b299-ca1d67f9a9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:20.727167 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464e6db-1130-4cf2-b299-ca1d67f9a9d0 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:20.727735 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464e6db-1130-4cf2-b299-ca1d67f9a9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:20.728237 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8464e6db-1130-4cf2-b299-ca1d67f9a9d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:20.741811 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8464e6db-1130-4cf2-b299-ca1d67f9a9d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:20.743611 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8464e6db-1130-4cf2-b299-ca1d67f9a9d0 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:20.743791 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-d6853540-22c3-439c-9d1b-1bf0aa70bf84 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:20.745046 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-d6853540-22c3-439c-9d1b-1bf0aa70bf84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:20.766434 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-8464e6db-1130-4cf2-b299-ca1d67f9a9d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:20.766434 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:20.766434 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:20.766434 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:20.766434 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:20.766434 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:20.766434 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:20.766434 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:20.766434 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:20.766434 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:20.766434 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:20.766434 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:20.766434 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:20.766434 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:20.766434 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:20.767893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:20.767893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:20.767893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:20.767893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:20.767893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:20.767893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:20.767893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:20.767893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:20.767893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:20.767893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:20.767893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:20.767893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:20.767893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:20.767893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:20.767893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:20.767893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:20.767893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:20.769246 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:20.769246 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:20.769246 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:20.769246 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:20.770520 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 440/867] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:10:20.776355 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8464e6db-1130-4cf2-b299-ca1d67f9a9d0 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:20.815725 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:20.840427 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 428/868] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:20 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:10:20.863039 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6853540-22c3-439c-9d1b-1bf0aa70bf84 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:20.906921 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8464e6db-1130-4cf2-b299-ca1d67f9a9d0 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:20.935474 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:20.953520 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fa035-3759-4a20-805b-b782bb2b28da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:20.954263 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fa035-3759-4a20-805b-b782bb2b28da None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:20.954544 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fa035-3759-4a20-805b-b782bb2b28da None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:20.954817 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fa035-3759-4a20-805b-b782bb2b28da None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:20.956161 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068fa035-3759-4a20-805b-b782bb2b28da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:20.956474 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068fa035-3759-4a20-805b-b782bb2b28da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:20.958384 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068fa035-3759-4a20-805b-b782bb2b28da None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:20.975741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 441/869] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:10:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:20.980922 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fa035-3759-4a20-805b-b782bb2b28da None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:20.985395 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6853540-22c3-439c-9d1b-1bf0aa70bf84 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:21.023450 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e70c73-91c1-4f54-9575-799e20cc1dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:21.024128 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e70c73-91c1-4f54-9575-799e20cc1dd3 None admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:21.024503 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e70c73-91c1-4f54-9575-799e20cc1dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:21.024779 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e70c73-91c1-4f54-9575-799e20cc1dd3 None admin] PATH_INFO: `/v3/services` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:21.025658 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e70c73-91c1-4f54-9575-799e20cc1dd3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:21.027165 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e70c73-91c1-4f54-9575-799e20cc1dd3 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:21.047288 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 429/870] 10.222.0.50 () {60 vars in 1141 bytes} [Fri May 27 03:10:20 2022] POST /identity/v3/services => generated 251 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:10:21.098761 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:21.113744 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fa035-3759-4a20-805b-b782bb2b28da None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:21.162002 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c1ddfc-8999-41f6-8f09-0fe46bdcde19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:21.163268 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1ddfc-8999-41f6-8f09-0fe46bdcde19 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:21.163753 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1ddfc-8999-41f6-8f09-0fe46bdcde19 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:21.164221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1ddfc-8999-41f6-8f09-0fe46bdcde19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:21.172638 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c1ddfc-8999-41f6-8f09-0fe46bdcde19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:21.174512 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c1ddfc-8999-41f6-8f09-0fe46bdcde19 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:21.189962 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-69c1ddfc-8999-41f6-8f09-0fe46bdcde19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:21.189962 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:21.189962 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:21.189962 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:21.189962 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:21.189962 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:21.189962 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:21.189962 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:21.189962 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:21.189962 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:21.189962 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:21.189962 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:21.189962 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:10:21.189962 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:10:21.189962 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:10:21.191128 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:10:21.191128 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:21.191128 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:21.191128 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:21.191128 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:21.191128 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:21.191128 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:21.191128 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:21.191128 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:21.191128 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:21.191128 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:21.191128 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:21.191128 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:21.191128 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:21.191128 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:21.191128 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:21.191128 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:21.191995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:10:21.191995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:10:21.191995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:10:21.191995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:10:21.191995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:10:21.191995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:10:21.191995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:21.191995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:21.194655 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 442/871] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:10:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:21.201043 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e70c73-91c1-4f54-9575-799e20cc1dd3 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:21.293217 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1547c7d1-e9ed-405c-9069-5c325b1f2ddd None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:21.293399 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1547c7d1-e9ed-405c-9069-5c325b1f2ddd None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:21.293672 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1547c7d1-e9ed-405c-9069-5c325b1f2ddd None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:21.295339 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 443/872] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:21.301868 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7368e8-8805-4765-9e9d-c3c1d93534a4 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:21.302162 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7368e8-8805-4765-9e9d-c3c1d93534a4 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:21.302440 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7368e8-8805-4765-9e9d-c3c1d93534a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:21.319278 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e70c73-91c1-4f54-9575-799e20cc1dd3 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:21.322732 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-1a7368e8-8805-4765-9e9d-c3c1d93534a4 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:21.324143 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-1a7368e8-8805-4765-9e9d-c3c1d93534a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:21.340470 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:21.363475 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8f1ad-8284-4220-9aed-e3fa6175cfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:21.364148 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8f1ad-8284-4220-9aed-e3fa6175cfc9 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:21.364390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8f1ad-8284-4220-9aed-e3fa6175cfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:21.364687 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8f1ad-8284-4220-9aed-e3fa6175cfc9 None admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:21.365474 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e8f1ad-8284-4220-9aed-e3fa6175cfc9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:21.365773 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e8f1ad-8284-4220-9aed-e3fa6175cfc9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:21.367200 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e8f1ad-8284-4220-9aed-e3fa6175cfc9 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:21.380769 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 430/873] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 03:10:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:21.386547 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8f1ad-8284-4220-9aed-e3fa6175cfc9 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:21.416767 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 444/874] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:21 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:10:21.440648 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7368e8-8805-4765-9e9d-c3c1d93534a4 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:21.524242 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8f1ad-8284-4220-9aed-e3fa6175cfc9 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:21.526932 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:21.552710 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7368e8-8805-4765-9e9d-c3c1d93534a4 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:21.570805 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1fcf98-cb06-4abc-a006-5428f56f8512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:21.571791 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1fcf98-cb06-4abc-a006-5428f56f8512 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:21.572085 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1fcf98-cb06-4abc-a006-5428f56f8512 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:21.572446 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1fcf98-cb06-4abc-a006-5428f56f8512 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:21.573432 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1fcf98-cb06-4abc-a006-5428f56f8512 None admin] RBAC: Adding query filter params (role.id=b4c524dd7bb44e94bd3b42d86e1cdf42, scope.project.id=f8398941252346468483ea0c863122f4, user.id=3de53e2f0fba4e0fac70887067c1963d) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:21.574033 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1fcf98-cb06-4abc-a006-5428f56f8512 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:21.575812 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1fcf98-cb06-4abc-a006-5428f56f8512 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:21.582816 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 431/875] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:10:21 2022] GET /identity/v3/role_assignments?user.id=3de53e2f0fba4e0fac70887067c1963d&scope.project.id=f8398941252346468483ea0c863122f4&role.id=b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:21.597882 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb3b7d-1e75-4c75-9455-287c437473af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:21.598880 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb3b7d-1e75-4c75-9455-287c437473af None admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:21.599326 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb3b7d-1e75-4c75-9455-287c437473af None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:21.599743 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb3b7d-1e75-4c75-9455-287c437473af None admin] PATH_INFO: `/v3/services` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:21.600645 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eb3b7d-1e75-4c75-9455-287c437473af None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:21.602303 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eb3b7d-1e75-4c75-9455-287c437473af None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:21.624799 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 445/876] 10.222.0.50 () {60 vars in 1141 bytes} [Fri May 27 03:10:21 2022] POST /identity/v3/services => generated 269 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:10:23.013521 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e91b48e-4ce5-4f22-aaa3-7dd181ce780b None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:23.014057 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e91b48e-4ce5-4f22-aaa3-7dd181ce780b None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:23.014057 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e91b48e-4ce5-4f22-aaa3-7dd181ce780b None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:23.015691 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 432/877] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:23.020977 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292db3ad-248f-4f7a-afe0-16160099f4ac None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:23.021401 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292db3ad-248f-4f7a-afe0-16160099f4ac None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:23.021571 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292db3ad-248f-4f7a-afe0-16160099f4ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:23.038999 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-292db3ad-248f-4f7a-afe0-16160099f4ac None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:23.040163 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-292db3ad-248f-4f7a-afe0-16160099f4ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:23.118483 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 446/878] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:23 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:10:23.142955 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e91b48e-4ce5-4f22-aaa3-7dd181ce780b None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:23.190220 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:23.255932 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e91b48e-4ce5-4f22-aaa3-7dd181ce780b None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:23.291693 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d45ee0f-ded1-4b9d-a1dc-c8d1bc88b9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:23.293377 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d45ee0f-ded1-4b9d-a1dc-c8d1bc88b9a7 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:23.293865 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d45ee0f-ded1-4b9d-a1dc-c8d1bc88b9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:23.294334 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d45ee0f-ded1-4b9d-a1dc-c8d1bc88b9a7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:23.305916 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d45ee0f-ded1-4b9d-a1dc-c8d1bc88b9a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:23.314145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-6d45ee0f-ded1-4b9d-a1dc-c8d1bc88b9a7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:10:23.314145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:23.314145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:23.314145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:23.314145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:23.314145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:23.314145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:23.314145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:23.314145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:23.314145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:23.314145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:23.314145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:23.314145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:23.314145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:23.314145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:23.314145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:23.315055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:23.315055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:23.315055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:23.315055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:23.315055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:23.315055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:23.315055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:23.315055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:23.315055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:23.315055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:23.315055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:23.315055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:23.315055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:23.315055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:23.315055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:23.315055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:23.315947 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:23.315947 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:23.315947 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:23.315947 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:23.315947 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:23.315947 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:23.315947 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:23.315947 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:23.315947 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:23.315947 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:23.315947 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:10:23.315947 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:23.315947 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 433/879] 10.222.0.50 () {56 vars in 1120 bytes} [Fri May 27 03:10:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:23.321506 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292db3ad-248f-4f7a-afe0-16160099f4ac None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:23.386908 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:23.419768 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa277834-fe76-4c79-9fac-3a7d88dd120a None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:23.420122 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa277834-fe76-4c79-9fac-3a7d88dd120a None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:23.420320 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa277834-fe76-4c79-9fac-3a7d88dd120a None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:23.422009 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 434/880] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:23.427329 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d90d59-12aa-4196-a250-223811274e80 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:23.427616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d90d59-12aa-4196-a250-223811274e80 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:23.427893 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d90d59-12aa-4196-a250-223811274e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:23.435923 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292db3ad-248f-4f7a-afe0-16160099f4ac None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:23.445434 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-d2d90d59-12aa-4196-a250-223811274e80 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:23.446637 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-d2d90d59-12aa-4196-a250-223811274e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:23.473545 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860bcadb-c5a1-43cf-b700-9f1ee580cdfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:23.474716 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860bcadb-c5a1-43cf-b700-9f1ee580cdfe None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:23.475242 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860bcadb-c5a1-43cf-b700-9f1ee580cdfe None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:23.475708 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860bcadb-c5a1-43cf-b700-9f1ee580cdfe None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:23.476646 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860bcadb-c5a1-43cf-b700-9f1ee580cdfe None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:23.477265 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860bcadb-c5a1-43cf-b700-9f1ee580cdfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:23.478937 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860bcadb-c5a1-43cf-b700-9f1ee580cdfe None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:23.490116 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 447/881] 10.222.0.50 () {56 vars in 1115 bytes} [Fri May 27 03:10:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:23.495861 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860bcadb-c5a1-43cf-b700-9f1ee580cdfe None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:23.525070 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 435/882] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:23 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:10:23.542055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d90d59-12aa-4196-a250-223811274e80 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:23.575952 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:23.609270 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860bcadb-c5a1-43cf-b700-9f1ee580cdfe None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:23.633894 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:23.645252 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d90d59-12aa-4196-a250-223811274e80 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:23.646722 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ec112-69bd-4a88-b27c-ff157002537a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:23.647814 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ec112-69bd-4a88-b27c-ff157002537a None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:23.648308 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ec112-69bd-4a88-b27c-ff157002537a None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:23.648797 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ec112-69bd-4a88-b27c-ff157002537a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:23.662371 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482ec112-69bd-4a88-b27c-ff157002537a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:23.664243 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482ec112-69bd-4a88-b27c-ff157002537a None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:23.683646 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-482ec112-69bd-4a88-b27c-ff157002537a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:23.683646 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:23.683646 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:23.683646 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:23.683646 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:23.683646 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:23.683646 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:23.683646 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:23.683646 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:23.683646 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:23.683646 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:23.683646 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:23.683646 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:23.683646 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:23.683646 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:23.684624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:23.684624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:23.684624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:23.684624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:23.684624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:23.684624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:23.684624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:23.684624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:23.684624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:23.684624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:23.684624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:23.684624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:23.684624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:23.684624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:23.684624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:23.684624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:23.684624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:23.685558 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:23.685558 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:23.685558 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:23.685558 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:23.685802 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7dc2c1-6591-4d7a-afc2-12e8eaba6b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:23.686521 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7dc2c1-6591-4d7a-afc2-12e8eaba6b15 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:23.686807 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7dc2c1-6591-4d7a-afc2-12e8eaba6b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:23.687071 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7dc2c1-6591-4d7a-afc2-12e8eaba6b15 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:23.687754 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7dc2c1-6591-4d7a-afc2-12e8eaba6b15 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:23.693798 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7dc2c1-6591-4d7a-afc2-12e8eaba6b15 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:23.694135 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 448/883] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:10:23.695303 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ec112-69bd-4a88-b27c-ff157002537a None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:23.703116 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-1c7dc2c1-6591-4d7a-afc2-12e8eaba6b15 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 03:10:23.703116 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:23.703116 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:10:23.703116 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:10:23.703116 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:10:23.703116 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:10:23.703116 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:10:23.703116 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:23.703116 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 03:10:23.703116 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:23.703116 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:10:23.703116 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:23.703116 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:23.703116 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:23.703116 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:23.703116 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:23.703116 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:23.704422 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:23.704422 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:23.704422 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:23.704422 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:23.704422 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:23.704422 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:23.704422 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:10:23.704422 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:10:23.704422 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:10:23.704422 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:10:23.704422 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:23.704422 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:23.704422 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:23.704422 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:23.704422 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:23.704422 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:23.705494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:23.705494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:23.705494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:23.705494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:23.705494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:23.705494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:23.705494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:23.705494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:23.705494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:23.705494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:23.705494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:10:23.705494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:23.705494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 03:10:23.705494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:23.712722 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 436/884] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 03:10:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:10:23.714123 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7dc2c1-6591-4d7a-afc2-12e8eaba6b15 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:23.810739 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ec112-69bd-4a88-b27c-ff157002537a None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:23.813980 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:23.817517 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7dc2c1-6591-4d7a-afc2-12e8eaba6b15 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:23.848164 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431eed7f-ee1b-4496-813b-ffe440e91442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:23.849299 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431eed7f-ee1b-4496-813b-ffe440e91442 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:23.849802 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431eed7f-ee1b-4496-813b-ffe440e91442 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:23.850245 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431eed7f-ee1b-4496-813b-ffe440e91442 None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:23.851150 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431eed7f-ee1b-4496-813b-ffe440e91442 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:23.851627 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431eed7f-ee1b-4496-813b-ffe440e91442 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:23.853302 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431eed7f-ee1b-4496-813b-ffe440e91442 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:23.859314 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29ba85-c5d7-4c8a-a6c4-c34c8228248f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:23.859997 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29ba85-c5d7-4c8a-a6c4-c34c8228248f None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:23.860278 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29ba85-c5d7-4c8a-a6c4-c34c8228248f None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:23.860554 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29ba85-c5d7-4c8a-a6c4-c34c8228248f None admin] PATH_INFO: `/v3/services` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:23.861304 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec29ba85-c5d7-4c8a-a6c4-c34c8228248f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:23.861611 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec29ba85-c5d7-4c8a-a6c4-c34c8228248f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:23.863129 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec29ba85-c5d7-4c8a-a6c4-c34c8228248f None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:23.868482 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 449/885] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:10:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:23.870135 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6744793-33c8-4440-bdf0-5c509b68147f None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:23.870642 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6744793-33c8-4440-bdf0-5c509b68147f None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:23.871115 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6744793-33c8-4440-bdf0-5c509b68147f None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:23.872741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 450/886] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:23.873060 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 437/887] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 03:10:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:23.874343 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6744793-33c8-4440-bdf0-5c509b68147f None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:23.877825 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ea316-a7dd-4f7c-8dcc-655efb7aaa2a None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:23.878229 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ea316-a7dd-4f7c-8dcc-655efb7aaa2a None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:23.878720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ea316-a7dd-4f7c-8dcc-655efb7aaa2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:23.896926 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-4e8ea316-a7dd-4f7c-8dcc-655efb7aaa2a None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:23.898141 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-4e8ea316-a7dd-4f7c-8dcc-655efb7aaa2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:23.911863 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:23.972548 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 438/888] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:23 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:10:23.973444 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8ea316-a7dd-4f7c-8dcc-655efb7aaa2a None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:23.979803 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6744793-33c8-4440-bdf0-5c509b68147f None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:24.016165 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdf86a6-e672-4970-8ffb-b90b831392d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:24.016795 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf86a6-e672-4970-8ffb-b90b831392d0 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:24.017176 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf86a6-e672-4970-8ffb-b90b831392d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:24.017449 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf86a6-e672-4970-8ffb-b90b831392d0 None admin] PATH_INFO: `/v3/services` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:24.018147 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdf86a6-e672-4970-8ffb-b90b831392d0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:24.018489 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdf86a6-e672-4970-8ffb-b90b831392d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:24.019865 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdf86a6-e672-4970-8ffb-b90b831392d0 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:24.024622 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 451/889] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 03:10:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:24.025833 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab5d57-6bf2-4bed-bdf7-6b7f88ed9733 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:24.026112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab5d57-6bf2-4bed-bdf7-6b7f88ed9733 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:24.026381 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ab5d57-6bf2-4bed-bdf7-6b7f88ed9733 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:24.027582 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 452/890] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:24.028265 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab5d57-6bf2-4bed-bdf7-6b7f88ed9733 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:24.067607 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:24.077879 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8ea316-a7dd-4f7c-8dcc-655efb7aaa2a None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:24.110586 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0250a940-4b0f-4685-b992-81e50445904e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:24.111401 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250a940-4b0f-4685-b992-81e50445904e None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:24.111673 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250a940-4b0f-4685-b992-81e50445904e None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:24.111942 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0250a940-4b0f-4685-b992-81e50445904e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:24.119192 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0250a940-4b0f-4685-b992-81e50445904e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:24.120664 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0250a940-4b0f-4685-b992-81e50445904e None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:24.128629 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:24.133007 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ab5d57-6bf2-4bed-bdf7-6b7f88ed9733 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:24.135450 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-0250a940-4b0f-4685-b992-81e50445904e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:10:24.135450 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:24.135450 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:24.135450 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:24.135450 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:24.135450 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:24.135450 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:24.135450 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:24.135450 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:24.135450 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:24.135450 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:24.135450 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:24.135450 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:10:24.135450 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:10:24.135450 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:10:24.135450 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:10:24.136521 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:24.136521 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:24.136521 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:10:24.136521 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:24.136521 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:10:24.136521 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:24.136521 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:24.136521 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:24.136521 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:24.136521 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:24.136521 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:24.136521 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:24.136521 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:24.136521 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:24.136521 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:24.136521 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:24.136521 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:24.136521 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:24.137699 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:24.137699 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:24.137699 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:10:24.137699 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:10:24.137699 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:10:24.137699 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:10:24.137699 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:10:24.137699 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:10:24.137699 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:10:24.137699 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:24.137699 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 439/891] 10.222.0.50 () {56 vars in 1117 bytes} [Fri May 27 03:10:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:24.137699 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0250a940-4b0f-4685-b992-81e50445904e None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:24.173069 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0888efac-2d95-4d5e-9dc9-11b1f3e3aa79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:24.173865 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888efac-2d95-4d5e-9dc9-11b1f3e3aa79 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:24.174142 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888efac-2d95-4d5e-9dc9-11b1f3e3aa79 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:24.174424 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888efac-2d95-4d5e-9dc9-11b1f3e3aa79 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:24.175710 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0888efac-2d95-4d5e-9dc9-11b1f3e3aa79 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:24.177099 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0888efac-2d95-4d5e-9dc9-11b1f3e3aa79 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:24.188866 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-0888efac-2d95-4d5e-9dc9-11b1f3e3aa79 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:10:24.188866 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:24.188866 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:10:24.188866 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:10:24.188866 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:10:24.188866 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:10:24.188866 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:10:24.188866 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:24.188866 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:10:24.188866 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:24.188866 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:10:24.188866 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:24.188866 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:24.188866 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:24.188866 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:24.188866 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:24.188866 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:24.189888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:24.189888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:24.189888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:24.189888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:24.189888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:24.189888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:24.189888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:10:24.189888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:10:24.189888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:10:24.189888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:10:24.189888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:24.189888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:24.189888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:24.189888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:24.189888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:24.189888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:24.191139 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:24.191139 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:24.191139 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:24.191139 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:24.191139 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:24.191139 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:24.191139 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:24.191139 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:24.191139 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:24.191139 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:24.191139 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:10:24.191139 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:24.191139 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:10:24.191139 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:24.191139 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 453/892] 10.222.0.50 () {56 vars in 1126 bytes} [Fri May 27 03:10:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:24.194324 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e86e8-70e1-419b-925d-48ceede37c8c None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:24.194591 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e86e8-70e1-419b-925d-48ceede37c8c None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:24.194877 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e86e8-70e1-419b-925d-48ceede37c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:24.212444 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-d55e86e8-70e1-419b-925d-48ceede37c8c None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:24.213982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-d55e86e8-70e1-419b-925d-48ceede37c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:24.245930 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0250a940-4b0f-4685-b992-81e50445904e None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:24.280377 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa38d3-f7cc-4c84-92be-ff817e8fe874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:24.281060 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa38d3-f7cc-4c84-92be-ff817e8fe874 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:24.281321 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa38d3-f7cc-4c84-92be-ff817e8fe874 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:24.281590 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa38d3-f7cc-4c84-92be-ff817e8fe874 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:24.282376 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffa38d3-f7cc-4c84-92be-ff817e8fe874 None admin] RBAC: Adding query filter params (interface=public, service_id=be32164beb26448ebc2ca0c388362eef, region_id=RegionOne) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:24.282589 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 454/893] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:24 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:10:24.282718 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffa38d3-f7cc-4c84-92be-ff817e8fe874 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:24.283432 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55e86e8-70e1-419b-925d-48ceede37c8c None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:24.284519 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffa38d3-f7cc-4c84-92be-ff817e8fe874 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:24.298014 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 440/894] 10.222.0.50 () {56 vars in 1251 bytes} [Fri May 27 03:10:24 2022] GET /identity/v3/endpoints?service_id=be32164beb26448ebc2ca0c388362eef&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:24.298014 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa38d3-f7cc-4c84-92be-ff817e8fe874 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:24.316512 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:24.385324 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:24.392897 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55e86e8-70e1-419b-925d-48ceede37c8c None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:24.401750 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa38d3-f7cc-4c84-92be-ff817e8fe874 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:24.428728 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e2c9e-e16c-4b6c-879b-94bf41272613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:24.429436 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e2c9e-e16c-4b6c-879b-94bf41272613 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:24.429696 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e2c9e-e16c-4b6c-879b-94bf41272613 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:24.429975 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e2c9e-e16c-4b6c-879b-94bf41272613 None admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:24.430795 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e2c9e-e16c-4b6c-879b-94bf41272613 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:24.431097 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e2c9e-e16c-4b6c-879b-94bf41272613 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:24.432527 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e2c9e-e16c-4b6c-879b-94bf41272613 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:24.437579 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0346b45-7e56-4e2f-a618-343d7bd78a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:24.438244 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0346b45-7e56-4e2f-a618-343d7bd78a22 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:24.438523 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0346b45-7e56-4e2f-a618-343d7bd78a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:24.438797 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0346b45-7e56-4e2f-a618-343d7bd78a22 None admin] PATH_INFO: `/v3/services` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:24.439502 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0346b45-7e56-4e2f-a618-343d7bd78a22 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:24.439802 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0346b45-7e56-4e2f-a618-343d7bd78a22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:24.441316 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0346b45-7e56-4e2f-a618-343d7bd78a22 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:24.445972 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 441/895] 10.222.0.50 () {56 vars in 1122 bytes} [Fri May 27 03:10:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:24.446740 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0346b45-7e56-4e2f-a618-343d7bd78a22 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:24.454294 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 455/896] 10.222.0.50 () {56 vars in 1149 bytes} [Fri May 27 03:10:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:24.455269 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e2c9e-e16c-4b6c-879b-94bf41272613 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:24.522695 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:24.544827 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0346b45-7e56-4e2f-a618-343d7bd78a22 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:24.556758 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e2c9e-e16c-4b6c-879b-94bf41272613 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:24.578424 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e9e8a-de70-4cf4-ad8f-8c657b1bde0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:24.579109 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e9e8a-de70-4cf4-ad8f-8c657b1bde0c None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:24.579387 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e9e8a-de70-4cf4-ad8f-8c657b1bde0c None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:24.579657 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e9e8a-de70-4cf4-ad8f-8c657b1bde0c None admin] PATH_INFO: `/v3/services` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:24.580331 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285e9e8a-de70-4cf4-ad8f-8c657b1bde0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:24.582141 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285e9e8a-de70-4cf4-ad8f-8c657b1bde0c None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:24.587772 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 442/897] 10.222.0.50 () {56 vars in 1087 bytes} [Fri May 27 03:10:24 2022] GET /identity/v3/services => generated 1268 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:10:24.588552 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e9e8a-de70-4cf4-ad8f-8c657b1bde0c None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:24.591255 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92c2c0-0b5f-4187-9629-487e5c2fa7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:24.592079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92c2c0-0b5f-4187-9629-487e5c2fa7ff None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:24.592353 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92c2c0-0b5f-4187-9629-487e5c2fa7ff None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:24.592620 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92c2c0-0b5f-4187-9629-487e5c2fa7ff None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:24.593309 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a92c2c0-0b5f-4187-9629-487e5c2fa7ff None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:24.594719 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a92c2c0-0b5f-4187-9629-487e5c2fa7ff None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:24.608046 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-4a92c2c0-0b5f-4187-9629-487e5c2fa7ff None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 03:10:24.608046 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:24.608046 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:10:24.608046 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:10:24.608046 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:10:24.608046 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:10:24.608046 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:10:24.608046 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:24.608046 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 03:10:24.608046 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:24.608046 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:10:24.608046 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:24.608046 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:24.608046 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:24.608046 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:24.608046 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:24.609094 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:24.609094 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:24.609094 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:24.609094 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:24.609094 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:24.609094 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:24.609094 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:24.609094 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:10:24.609094 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:10:24.609094 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:10:24.609094 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:10:24.609094 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:24.609094 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:24.609094 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:24.609094 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:24.609094 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:24.609938 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:24.609938 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:24.609938 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:24.609938 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:24.609938 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:24.609938 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:24.609938 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:24.609938 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:24.609938 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:24.609938 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:24.609938 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:24.609938 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:10:24.609938 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:24.609938 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 03:10:24.609938 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:24.609938 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 456/898] 10.222.0.50 () {56 vars in 1132 bytes} [Fri May 27 03:10:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:24.610773 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92c2c0-0b5f-4187-9629-487e5c2fa7ff None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:24.634313 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:24.720169 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e9e8a-de70-4cf4-ad8f-8c657b1bde0c None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:24.729221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92c2c0-0b5f-4187-9629-487e5c2fa7ff None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:24.757676 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dfd61d-8594-42ab-bcbd-6a5d42e0699a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:24.758351 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dfd61d-8594-42ab-bcbd-6a5d42e0699a None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:24.758625 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dfd61d-8594-42ab-bcbd-6a5d42e0699a None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:24.758890 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dfd61d-8594-42ab-bcbd-6a5d42e0699a None admin] PATH_INFO: `/v3/services` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:24.759600 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dfd61d-8594-42ab-bcbd-6a5d42e0699a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:24.759898 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dfd61d-8594-42ab-bcbd-6a5d42e0699a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:24.761426 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dfd61d-8594-42ab-bcbd-6a5d42e0699a None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:24.765118 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581d78f-8ef0-41cb-823f-2a6487415a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:24.765784 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581d78f-8ef0-41cb-823f-2a6487415a95 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:24.766059 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581d78f-8ef0-41cb-823f-2a6487415a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:24.766343 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581d78f-8ef0-41cb-823f-2a6487415a95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:24.766774 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 443/899] 10.222.0.50 () {56 vars in 1122 bytes} [Fri May 27 03:10:24 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:24.767617 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dfd61d-8594-42ab-bcbd-6a5d42e0699a None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:24.777509 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:24.867700 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8581d78f-8ef0-41cb-823f-2a6487415a95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:24.869309 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8581d78f-8ef0-41cb-823f-2a6487415a95 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:24.896540 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:24.911347 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dfd61d-8594-42ab-bcbd-6a5d42e0699a None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:24.963327 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab4eec7-0f8b-41b0-8aaa-9d8fd7051106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:24.964392 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4eec7-0f8b-41b0-8aaa-9d8fd7051106 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:24.965372 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4eec7-0f8b-41b0-8aaa-9d8fd7051106 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:24.965854 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4eec7-0f8b-41b0-8aaa-9d8fd7051106 None admin] PATH_INFO: `/v3/services` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:24.967202 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab4eec7-0f8b-41b0-8aaa-9d8fd7051106 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:24.967657 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab4eec7-0f8b-41b0-8aaa-9d8fd7051106 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:24.969269 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab4eec7-0f8b-41b0-8aaa-9d8fd7051106 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:24.976066 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 444/900] 10.222.0.50 () {56 vars in 1126 bytes} [Fri May 27 03:10:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:24.981699 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab4eec7-0f8b-41b0-8aaa-9d8fd7051106 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:24.994452 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 457/901] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:10:24 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:10:25.006102 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581d78f-8ef0-41cb-823f-2a6487415a95 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:25.081866 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:25.109727 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8581d78f-8ef0-41cb-823f-2a6487415a95 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:25.126382 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab4eec7-0f8b-41b0-8aaa-9d8fd7051106 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:25.128640 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:25.151512 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99624b3-10af-465c-9552-6408ded6a69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:25.152419 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99624b3-10af-465c-9552-6408ded6a69d None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:25.152696 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99624b3-10af-465c-9552-6408ded6a69d None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:25.152995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99624b3-10af-465c-9552-6408ded6a69d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:25.165873 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99624b3-10af-465c-9552-6408ded6a69d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:25.167524 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99624b3-10af-465c-9552-6408ded6a69d None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:25.167790 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3712fd9-2ab2-4936-bc54-14b2b2d29125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:25.168460 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3712fd9-2ab2-4936-bc54-14b2b2d29125 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:25.168736 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3712fd9-2ab2-4936-bc54-14b2b2d29125 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:25.169053 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3712fd9-2ab2-4936-bc54-14b2b2d29125 None admin] PATH_INFO: `/v3/services` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:25.169866 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3712fd9-2ab2-4936-bc54-14b2b2d29125 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:25.170274 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3712fd9-2ab2-4936-bc54-14b2b2d29125 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:25.171623 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3712fd9-2ab2-4936-bc54-14b2b2d29125 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:25.177115 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 445/902] 10.222.0.50 () {56 vars in 1126 bytes} [Fri May 27 03:10:24 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:25.182331 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3712fd9-2ab2-4936-bc54-14b2b2d29125 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:25.186421 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-b99624b3-10af-465c-9552-6408ded6a69d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:25.186421 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:25.186421 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:25.186421 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:25.186421 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:25.186421 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:25.186421 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:25.186421 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:25.186421 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:25.186421 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:25.186421 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:25.186421 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:25.186421 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:25.186421 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:25.186421 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:25.187333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:25.187333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:25.187333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:25.187333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:25.187333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:25.187333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:25.187333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:25.187333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:25.187333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:25.187333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:25.187333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:25.187333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:25.187333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:25.187333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:25.187333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:25.187333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:25.187333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:25.188219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:25.188219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:25.188219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:25.188219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:25.192307 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 458/903] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:10:25.193235 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99624b3-10af-465c-9552-6408ded6a69d None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:25.310498 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3712fd9-2ab2-4936-bc54-14b2b2d29125 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:25.322325 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99624b3-10af-465c-9552-6408ded6a69d None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:25.353323 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb96756-a7e7-4aef-b170-8868411b5c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:25.354259 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb96756-a7e7-4aef-b170-8868411b5c66 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:25.354551 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb96756-a7e7-4aef-b170-8868411b5c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:25.354842 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb96756-a7e7-4aef-b170-8868411b5c66 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:25.356299 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb96756-a7e7-4aef-b170-8868411b5c66 None admin] RBAC: Adding query filter params (interface=public, service_id=882f733bac5b4323855e8d5c3b931798, region_id=RegionOne) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:25.356617 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb96756-a7e7-4aef-b170-8868411b5c66 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:25.358654 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb96756-a7e7-4aef-b170-8868411b5c66 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:25.360834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:25.367219 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 446/904] 10.222.0.50 () {56 vars in 1251 bytes} [Fri May 27 03:10:25 2022] GET /identity/v3/endpoints?service_id=882f733bac5b4323855e8d5c3b931798&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:25.369518 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b727255-73f3-4180-a100-46355f646a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:25.370427 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b727255-73f3-4180-a100-46355f646a56 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:25.370921 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b727255-73f3-4180-a100-46355f646a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:25.371422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b727255-73f3-4180-a100-46355f646a56 None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:25.372023 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb96756-a7e7-4aef-b170-8868411b5c66 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:25.372568 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b727255-73f3-4180-a100-46355f646a56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:25.372936 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b727255-73f3-4180-a100-46355f646a56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:25.374458 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b727255-73f3-4180-a100-46355f646a56 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:25.394764 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 459/905] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:10:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:25.397461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b727255-73f3-4180-a100-46355f646a56 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:25.430939 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:25.496554 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb96756-a7e7-4aef-b170-8868411b5c66 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:25.516702 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b727255-73f3-4180-a100-46355f646a56 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:25.544902 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ab6fc-f536-40bb-8b65-ebe3f7ff2ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:25.546175 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ab6fc-f536-40bb-8b65-ebe3f7ff2ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:25.546873 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ab6fc-f536-40bb-8b65-ebe3f7ff2ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:25.547320 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ab6fc-f536-40bb-8b65-ebe3f7ff2ba5 None admin] PATH_INFO: `/v3/services` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:25.548877 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ab6fc-f536-40bb-8b65-ebe3f7ff2ba5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:25.550715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ab6fc-f536-40bb-8b65-ebe3f7ff2ba5 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:25.552733 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bc614c-578a-46a7-ae2f-497bf366b3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:25.553925 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc614c-578a-46a7-ae2f-497bf366b3dd None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:25.554261 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc614c-578a-46a7-ae2f-497bf366b3dd None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:25.554677 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bc614c-578a-46a7-ae2f-497bf366b3dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:25.558672 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 447/906] 10.222.0.50 () {56 vars in 1087 bytes} [Fri May 27 03:10:25 2022] GET /identity/v3/services => generated 1268 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:10:25.566497 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bc614c-578a-46a7-ae2f-497bf366b3dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:25.568064 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bc614c-578a-46a7-ae2f-497bf366b3dd None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:25.584285 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-60bc614c-578a-46a7-ae2f-497bf366b3dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:25.584285 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:25.584285 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:25.584285 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:25.584285 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:25.584285 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:25.584285 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:25.584285 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:25.584285 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:25.584285 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:25.584285 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:25.584285 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:25.584285 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:10:25.584285 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:10:25.584285 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:10:25.585320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:10:25.585320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:25.585320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:25.585320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:25.585320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:25.585320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:25.585320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:25.585320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:25.585320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:25.585320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:25.585320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:25.585320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:25.585320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:25.585320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:25.585320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:25.585320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:25.585320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:25.586209 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:10:25.586209 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:10:25.586209 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:10:25.586209 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:10:25.586209 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:10:25.586209 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:10:25.586209 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:25.586209 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:25.586209 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 460/907] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:10:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:25.593262 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ab6fc-f536-40bb-8b65-ebe3f7ff2ba5 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:25.683342 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:25.710725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ab6fc-f536-40bb-8b65-ebe3f7ff2ba5 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:25.749316 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f7f896-193f-4995-b3bf-1bc6e785c1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:25.750000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f7f896-193f-4995-b3bf-1bc6e785c1da None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:25.750281 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f7f896-193f-4995-b3bf-1bc6e785c1da None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:25.750555 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f7f896-193f-4995-b3bf-1bc6e785c1da None admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:25.751355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f7f896-193f-4995-b3bf-1bc6e785c1da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:25.751662 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f7f896-193f-4995-b3bf-1bc6e785c1da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:25.753664 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f7f896-193f-4995-b3bf-1bc6e785c1da None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:25.767785 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 448/908] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 03:10:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:25.774887 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bc614c-578a-46a7-ae2f-497bf366b3dd None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:25.870201 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:25.884403 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bc614c-578a-46a7-ae2f-497bf366b3dd None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:25.920618 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c728db16-d4e9-40ab-951b-3f4f1de2d2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:25.921399 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728db16-d4e9-40ab-951b-3f4f1de2d2e6 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:25.921681 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728db16-d4e9-40ab-951b-3f4f1de2d2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:25.922003 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728db16-d4e9-40ab-951b-3f4f1de2d2e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:25.922836 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c728db16-d4e9-40ab-951b-3f4f1de2d2e6 None admin] RBAC: Adding query filter params (role.id=8c199022919d4ef2b185859d8e1dc9a9, scope.project.id=f8398941252346468483ea0c863122f4, user.id=86a18e313e4944748aef58d54bfe7b13) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:25.923148 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c728db16-d4e9-40ab-951b-3f4f1de2d2e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:25.924658 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c728db16-d4e9-40ab-951b-3f4f1de2d2e6 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:25.931699 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 461/909] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:10:25 2022] GET /identity/v3/role_assignments?user.id=86a18e313e4944748aef58d54bfe7b13&scope.project.id=f8398941252346468483ea0c863122f4&role.id=8c199022919d4ef2b185859d8e1dc9a9 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:26.891445 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc5a8e4-3f14-45ea-8ae7-b0e6d0d5e6b7 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:26.891990 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc5a8e4-3f14-45ea-8ae7-b0e6d0d5e6b7 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:26.891990 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc5a8e4-3f14-45ea-8ae7-b0e6d0d5e6b7 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:26.893642 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 449/910] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:26.899154 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6d318-2cee-4e00-8f79-31de263e6d4a None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:26.899574 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6d318-2cee-4e00-8f79-31de263e6d4a None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:26.899740 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6d318-2cee-4e00-8f79-31de263e6d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:26.917311 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-e4a6d318-2cee-4e00-8f79-31de263e6d4a None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:26.918488 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-e4a6d318-2cee-4e00-8f79-31de263e6d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:26.986728 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 462/911] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:26 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:10:27.010570 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc5a8e4-3f14-45ea-8ae7-b0e6d0d5e6b7 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:27.025501 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfddd53-690f-44da-9381-3cd3df8d59cd None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:27.025794 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfddd53-690f-44da-9381-3cd3df8d59cd None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:27.026051 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfddd53-690f-44da-9381-3cd3df8d59cd None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:27.027671 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 463/912] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:10:27.037796 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c45f2-b2fa-4624-bd7d-fecad6ea57ce None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:27.038346 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c45f2-b2fa-4624-bd7d-fecad6ea57ce None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:27.038634 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454c45f2-b2fa-4624-bd7d-fecad6ea57ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:27.059081 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-454c45f2-b2fa-4624-bd7d-fecad6ea57ce None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:27.060714 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-454c45f2-b2fa-4624-bd7d-fecad6ea57ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:27.124524 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc5a8e4-3f14-45ea-8ae7-b0e6d0d5e6b7 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:27.148366 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:27.157088 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 464/913] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:27 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:10:27.168075 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e54685f-8201-402f-bdfd-c0b6e2f64292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:27.168979 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54685f-8201-402f-bdfd-c0b6e2f64292 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:27.169281 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54685f-8201-402f-bdfd-c0b6e2f64292 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:27.169554 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54685f-8201-402f-bdfd-c0b6e2f64292 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:27.170269 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e54685f-8201-402f-bdfd-c0b6e2f64292 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:27.171651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e54685f-8201-402f-bdfd-c0b6e2f64292 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:27.180254 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c45f2-b2fa-4624-bd7d-fecad6ea57ce None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:27.186275 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-9e54685f-8201-402f-bdfd-c0b6e2f64292 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 03:10:27.186275 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:27.186275 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:10:27.186275 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:10:27.186275 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:10:27.186275 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:10:27.186275 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:10:27.186275 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:27.186275 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 03:10:27.186275 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:27.186275 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:10:27.186275 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:27.186275 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:27.186275 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:27.186275 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:27.186275 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:27.186275 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:27.187236 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:27.187236 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:27.187236 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:27.187236 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:27.187236 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:27.187236 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:27.187236 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:10:27.187236 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:10:27.187236 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:10:27.187236 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:10:27.187236 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:27.187236 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:27.187236 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:27.187236 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:27.187236 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:27.187236 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:27.188055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:27.188055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:27.188055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:27.188055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:27.188055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:27.188055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:27.188055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:27.188055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:27.188055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:27.188055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:27.188055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:10:27.188055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:27.188055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 03:10:27.188055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:27.188055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 450/914] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 03:10:27 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:27.196524 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e54685f-8201-402f-bdfd-c0b6e2f64292 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:27.201855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:27.295609 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454c45f2-b2fa-4624-bd7d-fecad6ea57ce None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:27.333509 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5a1b5e-1aa0-441d-ae80-9b3da4e10b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:27.333820 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e54685f-8201-402f-bdfd-c0b6e2f64292 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:27.334638 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a1b5e-1aa0-441d-ae80-9b3da4e10b3b None admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:27.335096 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a1b5e-1aa0-441d-ae80-9b3da4e10b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:27.335500 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a1b5e-1aa0-441d-ae80-9b3da4e10b3b None admin] PATH_INFO: `/v3/services` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:27.336483 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5a1b5e-1aa0-441d-ae80-9b3da4e10b3b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:27.338102 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5a1b5e-1aa0-441d-ae80-9b3da4e10b3b None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:27.361267 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 465/915] 10.222.0.50 () {60 vars in 1141 bytes} [Fri May 27 03:10:27 2022] POST /identity/v3/services => generated 241 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:10:27.383140 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac7c54-3f28-4367-b899-b3e33651e294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:27.383833 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac7c54-3f28-4367-b899-b3e33651e294 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:27.384109 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac7c54-3f28-4367-b899-b3e33651e294 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:27.384377 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac7c54-3f28-4367-b899-b3e33651e294 None admin] PATH_INFO: `/v3/services` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:27.385143 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ac7c54-3f28-4367-b899-b3e33651e294 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:27.385448 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ac7c54-3f28-4367-b899-b3e33651e294 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:27.386895 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ac7c54-3f28-4367-b899-b3e33651e294 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:27.393069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 451/916] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 03:10:27 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:27.399269 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5a1b5e-1aa0-441d-ae80-9b3da4e10b3b None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:27.488186 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:27.527735 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5a1b5e-1aa0-441d-ae80-9b3da4e10b3b None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:27.564938 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e0c0ed-4903-474e-9730-8efe1cc143b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:27.566262 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e0c0ed-4903-474e-9730-8efe1cc143b1 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:27.566819 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e0c0ed-4903-474e-9730-8efe1cc143b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:27.567300 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e0c0ed-4903-474e-9730-8efe1cc143b1 None admin] PATH_INFO: `/v3/services` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:27.568230 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e0c0ed-4903-474e-9730-8efe1cc143b1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:27.568729 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e0c0ed-4903-474e-9730-8efe1cc143b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:27.570371 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e0c0ed-4903-474e-9730-8efe1cc143b1 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:27.577570 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 466/917] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 03:10:27 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:27.583585 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac7c54-3f28-4367-b899-b3e33651e294 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:27.637197 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:27.694034 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac7c54-3f28-4367-b899-b3e33651e294 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:27.732819 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2766ec-0bb8-4851-aa0d-977a0045acca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:27.733526 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2766ec-0bb8-4851-aa0d-977a0045acca None admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:27.733809 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2766ec-0bb8-4851-aa0d-977a0045acca None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:27.734084 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2766ec-0bb8-4851-aa0d-977a0045acca None admin] PATH_INFO: `/v3/endpoints` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:27.734845 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2766ec-0bb8-4851-aa0d-977a0045acca None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:27.736219 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2766ec-0bb8-4851-aa0d-977a0045acca None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:27.759661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 452/918] 10.222.0.50 () {60 vars in 1144 bytes} [Fri May 27 03:10:27 2022] POST /identity/v3/endpoints => generated 346 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:10:27.893781 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69979266-c799-465e-8677-9c2f47119f30 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:27.894172 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69979266-c799-465e-8677-9c2f47119f30 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:27.894335 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69979266-c799-465e-8677-9c2f47119f30 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:27.896035 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 467/919] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:27.901575 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d86e9d-ebf7-4975-a12b-46fbc7854afd None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:27.902096 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d86e9d-ebf7-4975-a12b-46fbc7854afd None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:27.902201 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d86e9d-ebf7-4975-a12b-46fbc7854afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:27.920731 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-65d86e9d-ebf7-4975-a12b-46fbc7854afd None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:27.921987 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-65d86e9d-ebf7-4975-a12b-46fbc7854afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:28.001841 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 453/920] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:10:28.025120 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69979266-c799-465e-8677-9c2f47119f30 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:28.136528 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69979266-c799-465e-8677-9c2f47119f30 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:28.170906 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd162526-b40c-4b59-a531-7f1728ee7399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:28.171726 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd162526-b40c-4b59-a531-7f1728ee7399 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:28.171999 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd162526-b40c-4b59-a531-7f1728ee7399 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:28.172264 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd162526-b40c-4b59-a531-7f1728ee7399 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:28.172902 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd162526-b40c-4b59-a531-7f1728ee7399 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:28.174305 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd162526-b40c-4b59-a531-7f1728ee7399 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:28.177262 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:28.187329 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-dd162526-b40c-4b59-a531-7f1728ee7399 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 03:10:28.187329 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:28.187329 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:10:28.187329 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:10:28.187329 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:10:28.187329 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:10:28.187329 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:10:28.187329 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:28.187329 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 03:10:28.187329 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:28.187329 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:10:28.187329 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:28.187329 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:28.187329 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:28.187329 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:28.187329 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:28.188580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:28.188580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:28.188580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:28.188580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:28.188580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:28.188580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:28.188580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:28.188580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:10:28.188580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:10:28.188580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:10:28.188580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:10:28.188580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:28.188580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:28.188580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:28.188580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:28.188580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:28.189648 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:28.189648 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:28.189648 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:28.189648 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:28.189648 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:28.189648 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:28.189648 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:28.189648 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:28.189648 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:28.189648 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:28.189648 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:28.189648 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:10:28.189648 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:28.189648 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 03:10:28.189648 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:28.189648 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 468/921] 10.222.0.50 () {56 vars in 1132 bytes} [Fri May 27 03:10:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:28.198119 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d86e9d-ebf7-4975-a12b-46fbc7854afd None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:28.226559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:28.278158 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a712b-de5b-4bd2-89c7-4f5db4677d14 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:28.278754 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a712b-de5b-4bd2-89c7-4f5db4677d14 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:28.279285 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574a712b-de5b-4bd2-89c7-4f5db4677d14 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:28.281218 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 469/922] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:28.288986 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2ec61-3275-44ff-b797-fb78e98f78ea None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:28.288986 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2ec61-3275-44ff-b797-fb78e98f78ea None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:28.288986 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2ec61-3275-44ff-b797-fb78e98f78ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:28.303168 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d86e9d-ebf7-4975-a12b-46fbc7854afd None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:28.310724 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-ebc2ec61-3275-44ff-b797-fb78e98f78ea None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:28.312288 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-ebc2ec61-3275-44ff-b797-fb78e98f78ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:28.340316 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b6eb46-54f8-456d-8067-0e4d07c91034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:28.341004 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b6eb46-54f8-456d-8067-0e4d07c91034 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:28.341286 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b6eb46-54f8-456d-8067-0e4d07c91034 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:28.341555 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b6eb46-54f8-456d-8067-0e4d07c91034 None admin] PATH_INFO: `/v3/services` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:28.342268 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b6eb46-54f8-456d-8067-0e4d07c91034 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:28.342568 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b6eb46-54f8-456d-8067-0e4d07c91034 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:28.343942 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b6eb46-54f8-456d-8067-0e4d07c91034 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:28.349413 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 454/923] 10.222.0.50 () {56 vars in 1126 bytes} [Fri May 27 03:10:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:28.355381 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b6eb46-54f8-456d-8067-0e4d07c91034 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:28.401237 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 470/924] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:28 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:10:28.420425 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2ec61-3275-44ff-b797-fb78e98f78ea None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:28.459391 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:28.469168 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b6eb46-54f8-456d-8067-0e4d07c91034 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:28.505872 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f42ea4d-1fe1-42ba-b09c-743dddaf8785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:28.507009 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42ea4d-1fe1-42ba-b09c-743dddaf8785 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:28.507579 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42ea4d-1fe1-42ba-b09c-743dddaf8785 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:28.508044 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f42ea4d-1fe1-42ba-b09c-743dddaf8785 None admin] PATH_INFO: `/v3/services` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:28.508995 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f42ea4d-1fe1-42ba-b09c-743dddaf8785 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:28.509496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f42ea4d-1fe1-42ba-b09c-743dddaf8785 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:28.511213 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f42ea4d-1fe1-42ba-b09c-743dddaf8785 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:28.516980 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 455/925] 10.222.0.50 () {56 vars in 1126 bytes} [Fri May 27 03:10:28 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:28.522424 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2ec61-3275-44ff-b797-fb78e98f78ea None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:28.525992 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f42ea4d-1fe1-42ba-b09c-743dddaf8785 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:28.555855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:28.563006 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9109c-9a24-403f-8ac0-12a178af67de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:28.564096 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9109c-9a24-403f-8ac0-12a178af67de None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:28.564704 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9109c-9a24-403f-8ac0-12a178af67de None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:28.565314 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9109c-9a24-403f-8ac0-12a178af67de None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:28.576894 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f9109c-9a24-403f-8ac0-12a178af67de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:28.585855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-69f9109c-9a24-403f-8ac0-12a178af67de None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:10:28.585855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:28.585855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:28.585855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:28.585855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:28.585855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:28.585855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:28.585855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:28.585855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:28.585855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:28.585855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:28.585855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:28.585855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:28.585855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:28.585855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:28.585855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:28.586795 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:28.586795 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:28.586795 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:28.586795 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:28.586795 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:28.586795 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:28.586795 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:28.586795 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:28.586795 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:28.586795 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:28.586795 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:28.586795 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:28.586795 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:28.586795 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:28.586795 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:28.586795 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:28.587624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:28.587624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:28.587624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:28.587624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:28.587624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:28.587624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:28.587624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:28.587624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:28.587624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:28.587624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:28.587624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:10:28.587624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:28.590480 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 471/926] 10.222.0.50 () {56 vars in 1120 bytes} [Fri May 27 03:10:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:10:28.597002 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9109c-9a24-403f-8ac0-12a178af67de None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:28.641889 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f42ea4d-1fe1-42ba-b09c-743dddaf8785 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:28.687348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30718486-cfcb-427e-9d9e-a8a4f46e8d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:28.689018 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30718486-cfcb-427e-9d9e-a8a4f46e8d4f None admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:28.689523 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30718486-cfcb-427e-9d9e-a8a4f46e8d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:28.690281 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30718486-cfcb-427e-9d9e-a8a4f46e8d4f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:28.691461 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30718486-cfcb-427e-9d9e-a8a4f46e8d4f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:28.693456 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30718486-cfcb-427e-9d9e-a8a4f46e8d4f None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:28.708935 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9109c-9a24-403f-8ac0-12a178af67de None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:28.721188 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 456/927] 10.222.0.50 () {60 vars in 1144 bytes} [Fri May 27 03:10:28 2022] POST /identity/v3/endpoints => generated 347 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:10:28.762164 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e12e717-4cca-452d-8e61-9691f513b4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:28.763578 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e12e717-4cca-452d-8e61-9691f513b4dd None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:28.764052 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e12e717-4cca-452d-8e61-9691f513b4dd None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:28.764517 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e12e717-4cca-452d-8e61-9691f513b4dd None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:28.765554 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e12e717-4cca-452d-8e61-9691f513b4dd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:28.766102 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e12e717-4cca-452d-8e61-9691f513b4dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:28.767930 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e12e717-4cca-452d-8e61-9691f513b4dd None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:28.778465 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 472/928] 10.222.0.50 () {56 vars in 1115 bytes} [Fri May 27 03:10:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:28.785355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30718486-cfcb-427e-9d9e-a8a4f46e8d4f None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:28.806115 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:28.900070 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30718486-cfcb-427e-9d9e-a8a4f46e8d4f None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:28.939996 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d728dac3-5aac-4042-b574-5019e05cc376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:28.941008 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d728dac3-5aac-4042-b574-5019e05cc376 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:28.941295 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d728dac3-5aac-4042-b574-5019e05cc376 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:28.941575 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d728dac3-5aac-4042-b574-5019e05cc376 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:28.952821 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d728dac3-5aac-4042-b574-5019e05cc376 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:28.954313 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d728dac3-5aac-4042-b574-5019e05cc376 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:28.973126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-d728dac3-5aac-4042-b574-5019e05cc376 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:28.973126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:28.973126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:28.973126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:28.973126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:28.973126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:28.973126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:28.973126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:28.973126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:28.973126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:28.973126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:28.973126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:28.973126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:28.973126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:28.973126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:28.974069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:28.974069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:28.974069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:28.974069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:28.974069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:28.974069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:28.974069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:28.974069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:28.974069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:28.974069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:28.974069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:28.974069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:28.974069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:28.974069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:28.974069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:28.974069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:28.974069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:28.974932 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:28.974932 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:28.974932 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:28.974932 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:28.974932 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 457/929] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:28.981445 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e12e717-4cca-452d-8e61-9691f513b4dd None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:29.036866 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:29.092574 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e12e717-4cca-452d-8e61-9691f513b4dd None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:29.130777 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428c36e-d31f-4711-ae34-b0357f8cbd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:29.132061 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428c36e-d31f-4711-ae34-b0357f8cbd1c None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:29.132591 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428c36e-d31f-4711-ae34-b0357f8cbd1c None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:29.133084 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428c36e-d31f-4711-ae34-b0357f8cbd1c None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:29.134022 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c428c36e-d31f-4711-ae34-b0357f8cbd1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:29.134521 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c428c36e-d31f-4711-ae34-b0357f8cbd1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:29.136095 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c428c36e-d31f-4711-ae34-b0357f8cbd1c None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:29.150101 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 473/930] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:10:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:29.156046 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d728dac3-5aac-4042-b574-5019e05cc376 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:29.225867 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:29.267171 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d728dac3-5aac-4042-b574-5019e05cc376 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:29.306578 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa35c2-7c98-441b-b58e-4613945eed3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:29.307884 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa35c2-7c98-441b-b58e-4613945eed3b None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:29.308364 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa35c2-7c98-441b-b58e-4613945eed3b None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:29.309027 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa35c2-7c98-441b-b58e-4613945eed3b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:29.317855 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfa35c2-7c98-441b-b58e-4613945eed3b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:29.319607 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfa35c2-7c98-441b-b58e-4613945eed3b None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:29.336661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-fdfa35c2-7c98-441b-b58e-4613945eed3b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:10:29.336661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:29.336661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:29.336661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:29.336661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:29.336661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:29.336661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:29.336661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:29.336661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:29.336661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:29.336661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:29.336661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:29.336661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:10:29.336661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:10:29.336661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:10:29.336661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:10:29.337668 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:29.337668 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:29.337668 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:10:29.337668 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:29.337668 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:10:29.337668 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:29.337668 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:29.337668 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:29.337668 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:29.337668 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:29.337668 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:29.337668 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:29.337668 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:29.337668 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:29.337668 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:29.337668 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:29.337668 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:29.337668 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:29.338580 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:29.338580 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:29.338580 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:10:29.338580 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:10:29.338580 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:10:29.338580 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:10:29.338580 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:10:29.338580 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:10:29.338580 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:10:29.338580 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:29.341431 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 458/931] 10.222.0.50 () {56 vars in 1117 bytes} [Fri May 27 03:10:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:29.347492 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428c36e-d31f-4711-ae34-b0357f8cbd1c None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:29.453609 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428c36e-d31f-4711-ae34-b0357f8cbd1c None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:29.464381 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:29.491595 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec1b286-82a1-4650-905e-066f58c02e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:29.492623 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1b286-82a1-4650-905e-066f58c02e1a None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:29.493140 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1b286-82a1-4650-905e-066f58c02e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:29.493604 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1b286-82a1-4650-905e-066f58c02e1a None admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:29.494639 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec1b286-82a1-4650-905e-066f58c02e1a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:29.495342 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec1b286-82a1-4650-905e-066f58c02e1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:29.497009 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec1b286-82a1-4650-905e-066f58c02e1a None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:29.515850 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 474/932] 10.222.0.50 () {56 vars in 1149 bytes} [Fri May 27 03:10:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:29.522116 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa35c2-7c98-441b-b58e-4613945eed3b None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:29.634876 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa35c2-7c98-441b-b58e-4613945eed3b None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:29.664298 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:29.677455 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d102ab3-85e2-44e3-aa37-9e7facbebd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:29.678270 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d102ab3-85e2-44e3-aa37-9e7facbebd72 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:29.678550 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d102ab3-85e2-44e3-aa37-9e7facbebd72 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:29.678816 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d102ab3-85e2-44e3-aa37-9e7facbebd72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:29.691224 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d102ab3-85e2-44e3-aa37-9e7facbebd72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:29.692780 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d102ab3-85e2-44e3-aa37-9e7facbebd72 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:29.711311 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-5d102ab3-85e2-44e3-aa37-9e7facbebd72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:29.711311 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:29.711311 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:29.711311 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:29.711311 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:29.711311 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:29.711311 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:29.711311 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:29.711311 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:29.711311 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:29.711311 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:29.711311 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:29.711311 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:29.711311 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:29.711311 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:29.712230 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:29.712230 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:29.712230 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:29.712230 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:29.712230 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:29.712230 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:29.712230 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:29.712230 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:29.712230 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:29.712230 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:29.712230 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:29.712230 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:29.712230 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:29.712230 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:29.712230 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:29.712230 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:29.712230 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:29.713206 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:29.713206 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:29.713206 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:29.713206 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:29.713206 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 459/933] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:10:29.719543 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec1b286-82a1-4650-905e-066f58c02e1a None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:29.735838 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b72cd13-ee41-4705-9482-e896ccdd9a4f None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:29.736132 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b72cd13-ee41-4705-9482-e896ccdd9a4f None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:29.736394 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b72cd13-ee41-4705-9482-e896ccdd9a4f None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:29.738172 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 460/934] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:10:29.745543 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a16567-fc58-45c7-8555-80adaaf6fae8 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:29.745832 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a16567-fc58-45c7-8555-80adaaf6fae8 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:29.746113 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a16567-fc58-45c7-8555-80adaaf6fae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:29.764992 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-16a16567-fc58-45c7-8555-80adaaf6fae8 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:29.766287 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-16a16567-fc58-45c7-8555-80adaaf6fae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:29.826068 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec1b286-82a1-4650-905e-066f58c02e1a None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:29.845195 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 461/935] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:10:29.860896 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a16567-fc58-45c7-8555-80adaaf6fae8 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:29.862851 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90938a-918b-4260-8997-0f068953cd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:29.863778 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90938a-918b-4260-8997-0f068953cd45 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:29.864268 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90938a-918b-4260-8997-0f068953cd45 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:29.864732 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90938a-918b-4260-8997-0f068953cd45 None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:29.865665 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90938a-918b-4260-8997-0f068953cd45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:29.866171 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90938a-918b-4260-8997-0f068953cd45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:29.867758 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90938a-918b-4260-8997-0f068953cd45 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:29.880205 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 475/936] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:10:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:29.885923 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90938a-918b-4260-8997-0f068953cd45 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:29.923367 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:29.977377 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a16567-fc58-45c7-8555-80adaaf6fae8 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:29.987700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:30.005898 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90938a-918b-4260-8997-0f068953cd45 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:30.022435 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41ced65-b80f-453a-a52b-1ecbd2b432f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:30.023262 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ced65-b80f-453a-a52b-1ecbd2b432f4 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:30.023541 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ced65-b80f-453a-a52b-1ecbd2b432f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:30.023839 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ced65-b80f-453a-a52b-1ecbd2b432f4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:30.024479 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41ced65-b80f-453a-a52b-1ecbd2b432f4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:30.025882 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41ced65-b80f-453a-a52b-1ecbd2b432f4 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:30.038846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-a41ced65-b80f-453a-a52b-1ecbd2b432f4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:10:30.038846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:30.038846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:10:30.038846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:10:30.038846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:10:30.038846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:10:30.038846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:10:30.038846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:30.038846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:10:30.038846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:30.038846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:10:30.038846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:30.038846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:30.038846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:30.038846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:30.038846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:30.038846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:30.039871 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:30.039871 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:30.039871 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:30.039871 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:30.039871 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:30.039871 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:30.039871 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:10:30.039871 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:10:30.039871 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:10:30.039871 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:10:30.039871 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:30.039871 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:30.039871 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:30.039871 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:30.039871 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:30.039871 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:30.040700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:30.040700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:30.040700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:30.040700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:30.040700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:30.040700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:30.040700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:30.040700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:30.040700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:30.040700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:30.040700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:10:30.040700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:30.040700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:10:30.040700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:30.040700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 462/937] 10.222.0.50 () {56 vars in 1126 bytes} [Fri May 27 03:10:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:10:30.048996 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41ced65-b80f-453a-a52b-1ecbd2b432f4 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:30.058250 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b6efa4-5225-4b6b-b765-ea06c8e2da20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:30.059060 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6efa4-5225-4b6b-b765-ea06c8e2da20 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:30.059339 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6efa4-5225-4b6b-b765-ea06c8e2da20 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:30.059610 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6efa4-5225-4b6b-b765-ea06c8e2da20 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:30.068021 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b6efa4-5225-4b6b-b765-ea06c8e2da20 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:30.069545 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b6efa4-5225-4b6b-b765-ea06c8e2da20 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:30.084551 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-33b6efa4-5225-4b6b-b765-ea06c8e2da20 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:30.084551 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:30.084551 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:30.084551 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:30.084551 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:30.084551 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:30.084551 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:30.084551 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:30.084551 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:30.084551 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:30.084551 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:30.084551 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:30.084551 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:10:30.084551 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:10:30.084551 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:10:30.085712 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:10:30.085712 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:30.085712 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:30.085712 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:30.085712 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:30.085712 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:30.085712 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:30.085712 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:30.085712 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:30.085712 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:30.085712 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:30.085712 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:30.085712 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:30.085712 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:30.085712 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:30.085712 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:30.085712 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:30.086586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:10:30.086586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:10:30.086586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:10:30.086586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:10:30.086586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:10:30.086586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:10:30.086586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:30.086586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:30.090725 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 476/938] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:10:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:10:30.091987 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b6efa4-5225-4b6b-b765-ea06c8e2da20 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:30.177756 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41ced65-b80f-453a-a52b-1ecbd2b432f4 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:30.177994 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:30.194924 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b6efa4-5225-4b6b-b765-ea06c8e2da20 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:30.218109 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb7a9b-2508-411b-bc4c-5a1c2a9814c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:30.219173 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb7a9b-2508-411b-bc4c-5a1c2a9814c6 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:30.219685 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb7a9b-2508-411b-bc4c-5a1c2a9814c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:30.220162 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb7a9b-2508-411b-bc4c-5a1c2a9814c6 None admin] PATH_INFO: `/v3/services` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:30.221074 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdb7a9b-2508-411b-bc4c-5a1c2a9814c6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:30.221685 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdb7a9b-2508-411b-bc4c-5a1c2a9814c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:30.223362 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdb7a9b-2508-411b-bc4c-5a1c2a9814c6 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:30.229040 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 463/939] 10.222.0.50 () {56 vars in 1122 bytes} [Fri May 27 03:10:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:30.230465 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972fb2fa-45aa-4908-b938-b7bdf0d69494 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:30.230748 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972fb2fa-45aa-4908-b938-b7bdf0d69494 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:30.231023 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972fb2fa-45aa-4908-b938-b7bdf0d69494 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:30.232464 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 464/940] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:30.234464 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972fb2fa-45aa-4908-b938-b7bdf0d69494 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:30.235423 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d1c374-76a5-42e4-94db-d6462bc92e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:30.236507 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d1c374-76a5-42e4-94db-d6462bc92e03 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:30.236794 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d1c374-76a5-42e4-94db-d6462bc92e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:30.237190 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d1c374-76a5-42e4-94db-d6462bc92e03 None admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:30.238079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d1c374-76a5-42e4-94db-d6462bc92e03 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:30.238395 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d1c374-76a5-42e4-94db-d6462bc92e03 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:30.239849 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d1c374-76a5-42e4-94db-d6462bc92e03 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:30.252642 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 477/941] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 03:10:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:30.254119 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f3056-7331-4d40-807b-80ab0c61e450 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:30.254409 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f3056-7331-4d40-807b-80ab0c61e450 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:30.254684 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5f3056-7331-4d40-807b-80ab0c61e450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:30.271440 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-7f5f3056-7331-4d40-807b-80ab0c61e450 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:30.272631 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-7f5f3056-7331-4d40-807b-80ab0c61e450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:30.284617 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:30.350051 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 478/942] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:10:30.351116 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5f3056-7331-4d40-807b-80ab0c61e450 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:30.375089 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972fb2fa-45aa-4908-b938-b7bdf0d69494 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:30.412674 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8fffad-aa1c-4aa7-9733-b29c815b70ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:30.413359 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8fffad-aa1c-4aa7-9733-b29c815b70ca None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:30.413643 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8fffad-aa1c-4aa7-9733-b29c815b70ca None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:30.413921 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8fffad-aa1c-4aa7-9733-b29c815b70ca None admin] PATH_INFO: `/v3/services` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:30.414638 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8fffad-aa1c-4aa7-9733-b29c815b70ca None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:30.414946 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8fffad-aa1c-4aa7-9733-b29c815b70ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:30.416321 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8fffad-aa1c-4aa7-9733-b29c815b70ca None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:30.421513 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 465/943] 10.222.0.50 () {56 vars in 1122 bytes} [Fri May 27 03:10:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:30.422309 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8fffad-aa1c-4aa7-9733-b29c815b70ca None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:30.452892 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:30.482502 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5f3056-7331-4d40-807b-80ab0c61e450 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:30.521802 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:30.523961 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128d948-1147-48f4-a1ca-87c06faabc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:30.524925 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128d948-1147-48f4-a1ca-87c06faabc4d None admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:30.525293 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128d948-1147-48f4-a1ca-87c06faabc4d None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:30.525562 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128d948-1147-48f4-a1ca-87c06faabc4d None admin] PATH_INFO: `/v3/projects/f8398941252346468483ea0c863122f4/users/86a18e313e4944748aef58d54bfe7b13/roles/8c199022919d4ef2b185859d8e1dc9a9` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:30.526180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8fffad-aa1c-4aa7-9733-b29c815b70ca None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:30.527877 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9128d948-1147-48f4-a1ca-87c06faabc4d None admin] RBAC: Authorizing `identity:create_grant(project_id=f8398941252346468483ea0c863122f4, user_id=86a18e313e4944748aef58d54bfe7b13, role_id=8c199022919d4ef2b185859d8e1dc9a9)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:30.529329 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9128d948-1147-48f4-a1ca-87c06faabc4d None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:30.544076 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 479/944] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 03:10:30 2022] PUT /identity/v3/projects/f8398941252346468483ea0c863122f4/users/86a18e313e4944748aef58d54bfe7b13/roles/8c199022919d4ef2b185859d8e1dc9a9 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:10:30.545254 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128d948-1147-48f4-a1ca-87c06faabc4d None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:30.576139 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b3491-d271-4290-855a-6cb59c4ae999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:30.577271 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b3491-d271-4290-855a-6cb59c4ae999 None admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:30.577776 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b3491-d271-4290-855a-6cb59c4ae999 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:30.578291 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111b3491-d271-4290-855a-6cb59c4ae999 None admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:30.579327 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111b3491-d271-4290-855a-6cb59c4ae999 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:30.581278 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111b3491-d271-4290-855a-6cb59c4ae999 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:30.583206 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: WARNING keystone.server.flask.common [None req-111b3491-d271-4290-855a-6cb59c4ae999 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 27 03:10:30.609055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.sql.core [None req-111b3491-d271-4290-855a-6cb59c4ae999 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 27 03:10:30.609055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 27 03:10:30.609055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [parameters: {'id': 'ff3707affeba4a2f9cd11d3ebb8d9214', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 27 03:10:30.609055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80410) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 27 03:10:30.624915 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-111b3491-d271-4290-855a-6cb59c4ae999 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ff3707affeba4a2f9cd11d3ebb8d9214.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ff3707affeba4a2f9cd11d3ebb8d9214. May 27 03:10:30.624915 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:30.624915 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 03:10:30.624915 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 03:10:30.624915 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 03:10:30.624915 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 03:10:30.624915 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 03:10:30.624915 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application result = self._query(query) May 27 03:10:30.624915 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 03:10:30.624915 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application conn.query(q) May 27 03:10:30.624915 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 03:10:30.624915 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 03:10:30.624915 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 03:10:30.625846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application result.read() May 27 03:10:30.625846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 03:10:30.625846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 03:10:30.625846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 03:10:30.625846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 03:10:30.625846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 03:10:30.625846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 03:10:30.625846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 03:10:30.625846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 03:10:30.625846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 27 03:10:30.625846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:30.625846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 27 03:10:30.625846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:30.625846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:30.625846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 27 03:10:30.625846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 27 03:10:30.625846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 27 03:10:30.625846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return project_ref.to_dict() May 27 03:10:30.626789 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 03:10:30.626789 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application next(self.gen) May 27 03:10:30.626789 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 27 03:10:30.626789 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application yield resource May 27 03:10:30.626789 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 03:10:30.626789 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application next(self.gen) May 27 03:10:30.626789 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 27 03:10:30.626789 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self.session.rollback() May 27 03:10:30.626789 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 27 03:10:30.626789 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self.force_reraise() May 27 03:10:30.626789 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 27 03:10:30.626789 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise self.value May 27 03:10:30.626789 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 27 03:10:30.626789 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 27 03:10:30.626789 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 27 03:10:30.626789 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application session.commit() May 27 03:10:30.626789 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 27 03:10:30.627817 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 27 03:10:30.627817 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 27 03:10:30.627817 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self._prepare_impl() May 27 03:10:30.627817 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 27 03:10:30.627817 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self.session.flush() May 27 03:10:30.627817 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 27 03:10:30.627817 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self._flush(objects) May 27 03:10:30.627817 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 27 03:10:30.627817 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 27 03:10:30.627817 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 27 03:10:30.627817 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application compat.raise_( May 27 03:10:30.627817 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 03:10:30.627817 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception May 27 03:10:30.627817 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 27 03:10:30.627817 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application flush_context.execute() May 27 03:10:30.627817 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 27 03:10:30.627817 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rec.execute(self) May 27 03:10:30.629183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 27 03:10:30.629183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 27 03:10:30.629183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 27 03:10:30.629183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application _emit_insert_statements( May 27 03:10:30.629183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 27 03:10:30.629183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application c = connection._execute_20( May 27 03:10:30.629183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 27 03:10:30.629183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 27 03:10:30.629183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 27 03:10:30.629183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 27 03:10:30.629183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 27 03:10:30.629183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ret = self._execute_context( May 27 03:10:30.629183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 27 03:10:30.629183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 27 03:10:30.629183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 27 03:10:30.631080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 27 03:10:30.631080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 03:10:30.631080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception May 27 03:10:30.631080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 03:10:30.631080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 03:10:30.631080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 03:10:30.631080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 03:10:30.631080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 03:10:30.631080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application result = self._query(query) May 27 03:10:30.631080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 03:10:30.631080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application conn.query(q) May 27 03:10:30.631080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 03:10:30.631080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 03:10:30.631080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 03:10:30.631080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application result.read() May 27 03:10:30.631080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 03:10:30.631080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 03:10:30.633326 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 03:10:30.633326 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 03:10:30.633326 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 03:10:30.633326 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 03:10:30.633326 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 03:10:30.633326 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 03:10:30.633326 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 27 03:10:30.633326 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 27 03:10:30.633326 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [parameters: {'id': 'ff3707affeba4a2f9cd11d3ebb8d9214', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 27 03:10:30.633326 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 27 03:10:30.633326 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:30.633326 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:10:30.633326 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:30.633326 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:30.633326 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 27 03:10:30.635276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 27 03:10:30.635276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 27 03:10:30.635276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 27 03:10:30.635276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 27 03:10:30.635276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:30.635276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:10:30.635276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:30.635276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:30.635276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:30.635276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:30.635276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:30.635276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:30.635276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:30.635276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:30.635276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:30.635276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:30.635276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:30.638522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:30.638522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 27 03:10:30.638522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 27 03:10:30.638522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:30.638522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:30.638522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 27 03:10:30.638522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.Conflict( May 27 03:10:30.638522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id ff3707affeba4a2f9cd11d3ebb8d9214. May 27 03:10:30.638522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:30.638522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 466/945] 10.222.0.50 () {60 vars in 1140 bytes} [Fri May 27 03:10:30 2022] POST /identity/v3/projects => generated 262 bytes in 216 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:30.640395 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b3491-d271-4290-855a-6cb59c4ae999 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:30.696138 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128d948-1147-48f4-a1ca-87c06faabc4d None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:30.736489 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd3c502-3d18-4b02-ae84-c119f448511f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:30.737740 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3c502-3d18-4b02-ae84-c119f448511f None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:30.737740 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3c502-3d18-4b02-ae84-c119f448511f None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:30.737998 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3c502-3d18-4b02-ae84-c119f448511f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:30.738631 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd3c502-3d18-4b02-ae84-c119f448511f None admin] RBAC: Adding query filter params (interface=public, service_id=2a7fc41176514aa1b8623624394cb650, region_id=RegionOne) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:30.739162 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd3c502-3d18-4b02-ae84-c119f448511f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:30.740642 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd3c502-3d18-4b02-ae84-c119f448511f None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:30.744082 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:30.748622 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 480/946] 10.222.0.50 () {56 vars in 1251 bytes} [Fri May 27 03:10:30 2022] GET /identity/v3/endpoints?service_id=2a7fc41176514aa1b8623624394cb650&interface=public®ion_id=RegionOne => generated 194 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:30.755594 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd3c502-3d18-4b02-ae84-c119f448511f None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:30.793525 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111b3491-d271-4290-855a-6cb59c4ae999 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:30.835117 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc70e58-6526-41f1-8e20-c9e2ab41e9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:30.836313 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc70e58-6526-41f1-8e20-c9e2ab41e9dd None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:30.836733 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc70e58-6526-41f1-8e20-c9e2ab41e9dd None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:30.837206 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc70e58-6526-41f1-8e20-c9e2ab41e9dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:30.847383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc70e58-6526-41f1-8e20-c9e2ab41e9dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:30.850200 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc70e58-6526-41f1-8e20-c9e2ab41e9dd None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:30.870806 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-0bc70e58-6526-41f1-8e20-c9e2ab41e9dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:30.870806 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:30.870806 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:30.870806 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:30.870806 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:30.870806 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:30.870806 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:30.870806 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:30.870806 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:30.870806 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:30.870806 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:30.870806 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:30.870806 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:10:30.870806 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:10:30.870806 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:10:30.871922 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:10:30.871922 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:30.871922 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:30.871922 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:30.871922 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:30.871922 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:30.871922 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:30.871922 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:30.871922 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:30.871922 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:30.871922 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:30.871922 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:30.871922 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:30.871922 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:30.871922 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:30.871922 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:30.871922 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:30.872849 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:10:30.872849 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:10:30.872849 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:10:30.872849 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:10:30.872849 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:10:30.872849 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:10:30.872849 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:30.872849 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:30.872849 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 467/947] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:10:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:30.879672 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc70e58-6526-41f1-8e20-c9e2ab41e9dd None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:30.882869 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd3c502-3d18-4b02-ae84-c119f448511f None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:30.897364 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:30.930640 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a35cd3-2f3b-41c1-8b05-59e9bfb49628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:30.932069 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a35cd3-2f3b-41c1-8b05-59e9bfb49628 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:30.932588 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a35cd3-2f3b-41c1-8b05-59e9bfb49628 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:30.933024 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a35cd3-2f3b-41c1-8b05-59e9bfb49628 None admin] PATH_INFO: `/v3/services` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:30.934117 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a35cd3-2f3b-41c1-8b05-59e9bfb49628 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:30.936383 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a35cd3-2f3b-41c1-8b05-59e9bfb49628 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:30.945080 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 481/948] 10.222.0.50 () {56 vars in 1087 bytes} [Fri May 27 03:10:30 2022] GET /identity/v3/services => generated 1497 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:10:31.007199 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc70e58-6526-41f1-8e20-c9e2ab41e9dd None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:31.048123 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de472c53-7443-4102-ab48-98cf12b2b4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:31.048981 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de472c53-7443-4102-ab48-98cf12b2b4ce None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:31.049447 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de472c53-7443-4102-ab48-98cf12b2b4ce None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:31.049872 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de472c53-7443-4102-ab48-98cf12b2b4ce None admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:31.050758 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de472c53-7443-4102-ab48-98cf12b2b4ce None admin] RBAC: Adding query filter params (name=service) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:31.051185 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de472c53-7443-4102-ab48-98cf12b2b4ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:31.052851 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de472c53-7443-4102-ab48-98cf12b2b4ce None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:31.076321 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 468/949] 10.222.0.50 () {56 vars in 1112 bytes} [Fri May 27 03:10:30 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:31.159979 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1f341-d607-47a4-bba5-0a6a495f2e43 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:31.160520 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1f341-d607-47a4-bba5-0a6a495f2e43 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:31.160520 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a1f341-d607-47a4-bba5-0a6a495f2e43 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:31.162195 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 482/950] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:31.168436 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adaa82b-d8ca-45cc-a259-364facea58d9 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:31.168875 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adaa82b-d8ca-45cc-a259-364facea58d9 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:31.169279 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adaa82b-d8ca-45cc-a259-364facea58d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:31.188837 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-2adaa82b-d8ca-45cc-a259-364facea58d9 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:31.190865 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-2adaa82b-d8ca-45cc-a259-364facea58d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:31.271404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 469/951] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:10:31.293787 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a1f341-d607-47a4-bba5-0a6a495f2e43 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:31.367536 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:31.407999 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a1f341-d607-47a4-bba5-0a6a495f2e43 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:31.443969 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a79a7f-3c48-4f51-9f43-8b9527792ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:31.444773 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a79a7f-3c48-4f51-9f43-8b9527792ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:31.445076 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a79a7f-3c48-4f51-9f43-8b9527792ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:31.445349 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a79a7f-3c48-4f51-9f43-8b9527792ca6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:31.445996 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a79a7f-3c48-4f51-9f43-8b9527792ca6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:31.447432 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a79a7f-3c48-4f51-9f43-8b9527792ca6 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:31.459753 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-51a79a7f-3c48-4f51-9f43-8b9527792ca6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 27 03:10:31.459753 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:31.459753 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:10:31.459753 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:10:31.459753 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:10:31.459753 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:10:31.459753 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:10:31.459753 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:31.459753 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 03:10:31.459753 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:31.459753 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:10:31.459753 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:31.459753 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:31.459753 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:31.459753 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:31.459753 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:31.459753 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:31.461221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:31.461221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:31.461221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:31.461221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:31.461221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:31.461221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:31.461221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:10:31.461221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:10:31.461221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:10:31.461221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:10:31.461221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:31.461221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:31.461221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:31.461221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:31.461221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:31.461221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:31.462319 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:31.462319 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:31.462319 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:31.462319 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:31.462319 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:31.462319 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:31.462319 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:31.462319 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:31.462319 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:31.462319 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:31.462319 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:10:31.462319 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:31.462319 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 03:10:31.462319 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:31.466515 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 483/952] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:10:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:31.467616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adaa82b-d8ca-45cc-a259-364facea58d9 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:31.471927 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:31.573505 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adaa82b-d8ca-45cc-a259-364facea58d9 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:31.609768 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea0cf3-51ce-48cc-9a82-486c0ac794a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:31.610439 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea0cf3-51ce-48cc-9a82-486c0ac794a5 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:31.610720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea0cf3-51ce-48cc-9a82-486c0ac794a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:31.610993 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea0cf3-51ce-48cc-9a82-486c0ac794a5 None admin] PATH_INFO: `/v3/services` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:31.611708 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bea0cf3-51ce-48cc-9a82-486c0ac794a5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:31.612115 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bea0cf3-51ce-48cc-9a82-486c0ac794a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:31.613536 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bea0cf3-51ce-48cc-9a82-486c0ac794a5 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:31.618996 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 470/953] 10.222.0.50 () {56 vars in 1112 bytes} [Fri May 27 03:10:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:31.624558 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a79a7f-3c48-4f51-9f43-8b9527792ca6 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:31.730907 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a79a7f-3c48-4f51-9f43-8b9527792ca6 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:31.761318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:31.770894 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a54c0c-d298-4f92-8440-0143a9e8b8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:31.771659 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a54c0c-d298-4f92-8440-0143a9e8b8f8 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:31.771933 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a54c0c-d298-4f92-8440-0143a9e8b8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:31.772200 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a54c0c-d298-4f92-8440-0143a9e8b8f8 None admin] PATH_INFO: `/v3/services` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:31.772901 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a54c0c-d298-4f92-8440-0143a9e8b8f8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:31.773227 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a54c0c-d298-4f92-8440-0143a9e8b8f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:31.774672 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a54c0c-d298-4f92-8440-0143a9e8b8f8 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:31.782892 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 484/954] 10.222.0.50 () {56 vars in 1112 bytes} [Fri May 27 03:10:31 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:32.790749 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25423b4b-b888-4e88-8bdd-851627c12b18 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:32.791324 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25423b4b-b888-4e88-8bdd-851627c12b18 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:32.791324 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25423b4b-b888-4e88-8bdd-851627c12b18 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:32.793120 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 471/955] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:32.798607 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844be5ed-d547-4626-a9e1-1e0a46b4d906 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:32.799009 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844be5ed-d547-4626-a9e1-1e0a46b4d906 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:32.799176 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844be5ed-d547-4626-a9e1-1e0a46b4d906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:32.816314 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-844be5ed-d547-4626-a9e1-1e0a46b4d906 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:32.817507 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-844be5ed-d547-4626-a9e1-1e0a46b4d906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:32.891923 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 485/956] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:10:32.916059 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25423b4b-b888-4e88-8bdd-851627c12b18 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:32.988109 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:33.024671 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25423b4b-b888-4e88-8bdd-851627c12b18 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:33.063217 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e275e-cec6-4995-bbb1-4ae3d244be81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:33.063984 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e275e-cec6-4995-bbb1-4ae3d244be81 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:33.064272 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e275e-cec6-4995-bbb1-4ae3d244be81 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:33.064540 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e275e-cec6-4995-bbb1-4ae3d244be81 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:33.077540 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e275e-cec6-4995-bbb1-4ae3d244be81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:33.086228 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-2c2e275e-cec6-4995-bbb1-4ae3d244be81 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:10:33.086228 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:33.086228 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:33.086228 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:33.086228 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:33.086228 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:33.086228 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:33.086228 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:33.086228 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:33.086228 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:33.086228 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:33.086228 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:33.086228 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:33.086228 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:33.086228 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:33.086228 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:33.087197 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:33.087197 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:33.087197 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:33.087197 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:33.087197 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:33.087197 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:33.087197 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:33.087197 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:33.087197 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:33.087197 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:33.087197 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:33.087197 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:33.087197 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:33.087197 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:33.087197 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:33.087197 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:33.088065 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:33.088065 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:33.088065 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:33.088065 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:33.088065 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:33.088065 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:33.088065 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:33.088065 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:33.088065 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:33.088065 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:33.088065 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:10:33.088065 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:33.088065 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 472/957] 10.222.0.50 () {56 vars in 1120 bytes} [Fri May 27 03:10:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:33.093421 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844be5ed-d547-4626-a9e1-1e0a46b4d906 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:33.195827 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844be5ed-d547-4626-a9e1-1e0a46b4d906 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:33.220820 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:33.232698 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed894c29-349c-483d-a0e8-a5fd0038736b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:33.233412 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed894c29-349c-483d-a0e8-a5fd0038736b None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:33.233698 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed894c29-349c-483d-a0e8-a5fd0038736b None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:33.233980 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed894c29-349c-483d-a0e8-a5fd0038736b None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:33.235091 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed894c29-349c-483d-a0e8-a5fd0038736b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:33.235765 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed894c29-349c-483d-a0e8-a5fd0038736b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:33.237800 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed894c29-349c-483d-a0e8-a5fd0038736b None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:33.246767 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 486/958] 10.222.0.50 () {56 vars in 1115 bytes} [Fri May 27 03:10:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:33.252149 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b0e18-8bee-48c7-a8b3-50267e36c623 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:33.252759 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b0e18-8bee-48c7-a8b3-50267e36c623 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:33.252759 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b0e18-8bee-48c7-a8b3-50267e36c623 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:33.254433 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 473/959] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:33.255231 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed894c29-349c-483d-a0e8-a5fd0038736b None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:33.259929 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f060dc43-8587-45ed-a222-dca835dce10b None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:33.260462 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f060dc43-8587-45ed-a222-dca835dce10b None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:33.260934 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f060dc43-8587-45ed-a222-dca835dce10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:33.279451 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-f060dc43-8587-45ed-a222-dca835dce10b None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:33.281756 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-f060dc43-8587-45ed-a222-dca835dce10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:33.366625 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 474/960] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:10:33.380448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed894c29-349c-483d-a0e8-a5fd0038736b None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:33.389273 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f060dc43-8587-45ed-a222-dca835dce10b None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:33.416587 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065d0f7-3c74-40da-a247-a0b0992a7947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:33.417473 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065d0f7-3c74-40da-a247-a0b0992a7947 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:33.417756 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065d0f7-3c74-40da-a247-a0b0992a7947 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:33.418027 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065d0f7-3c74-40da-a247-a0b0992a7947 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:33.429638 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065d0f7-3c74-40da-a247-a0b0992a7947 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:33.431161 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065d0f7-3c74-40da-a247-a0b0992a7947 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:33.448966 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-6065d0f7-3c74-40da-a247-a0b0992a7947 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:33.448966 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:33.448966 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:33.448966 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:33.448966 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:33.448966 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:33.448966 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:33.448966 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:33.448966 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:33.448966 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:33.448966 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:33.448966 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:33.448966 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:33.448966 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:33.448966 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:33.449929 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:33.449929 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:33.449929 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:33.449929 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:33.449929 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:33.449929 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:33.449929 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:33.449929 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:33.449929 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:33.449929 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:33.449929 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:33.449929 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:33.449929 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:33.449929 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:33.449929 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:33.449929 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:33.449929 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:33.450799 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:33.450799 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:33.450799 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:33.450799 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:33.451885 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:33.455038 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 487/961] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:33.456003 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065d0f7-3c74-40da-a247-a0b0992a7947 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:33.492528 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:33.496532 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f060dc43-8587-45ed-a222-dca835dce10b None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:33.535774 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95996489-da19-470f-814a-b199a9a9b28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:33.536591 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95996489-da19-470f-814a-b199a9a9b28c None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:33.537243 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95996489-da19-470f-814a-b199a9a9b28c None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:33.537534 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95996489-da19-470f-814a-b199a9a9b28c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:33.538197 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95996489-da19-470f-814a-b199a9a9b28c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:33.539898 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95996489-da19-470f-814a-b199a9a9b28c None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:33.552859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-95996489-da19-470f-814a-b199a9a9b28c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:10:33.552859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:33.552859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:10:33.552859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:10:33.552859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:10:33.552859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:10:33.552859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:10:33.552859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:33.552859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:10:33.552859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:33.552859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:10:33.552859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:33.552859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:33.552859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:33.552859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:33.552859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:33.552859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:33.553964 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:33.553964 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:33.553964 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:33.553964 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:33.553964 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:33.553964 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:33.553964 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:10:33.553964 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:10:33.553964 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:10:33.553964 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:10:33.553964 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:33.553964 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:33.553964 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:33.553964 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:33.553964 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:33.553964 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:33.554803 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:33.554803 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:33.554803 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:33.554803 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:33.554803 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:33.554803 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:33.554803 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:33.554803 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:33.554803 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:33.554803 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:33.554803 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:10:33.554803 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:33.554803 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:10:33.554803 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:33.554803 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 475/962] 10.222.0.50 () {56 vars in 1126 bytes} [Fri May 27 03:10:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:10:33.556242 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3feaa00-39b1-4ff7-bdf3-6d282b15925a None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:33.556242 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3feaa00-39b1-4ff7-bdf3-6d282b15925a None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:33.556242 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3feaa00-39b1-4ff7-bdf3-6d282b15925a None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:33.557626 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 476/963] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:33.562297 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3feaa00-39b1-4ff7-bdf3-6d282b15925a None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:33.568549 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065d0f7-3c74-40da-a247-a0b0992a7947 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:33.602591 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c750510a-aeca-45fa-9ff0-fa822e5c3530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:33.603360 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750510a-aeca-45fa-9ff0-fa822e5c3530 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:33.603644 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750510a-aeca-45fa-9ff0-fa822e5c3530 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:33.603919 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750510a-aeca-45fa-9ff0-fa822e5c3530 None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:33.604645 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c750510a-aeca-45fa-9ff0-fa822e5c3530 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:33.604946 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c750510a-aeca-45fa-9ff0-fa822e5c3530 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:33.606488 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c750510a-aeca-45fa-9ff0-fa822e5c3530 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:33.619070 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 488/964] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:10:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:33.623131 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd79b5-7ce7-4216-a1fb-ef2d5477739a None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:33.623664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd79b5-7ce7-4216-a1fb-ef2d5477739a None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:33.624180 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd79b5-7ce7-4216-a1fb-ef2d5477739a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:33.642195 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-addd79b5-7ce7-4216-a1fb-ef2d5477739a None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:33.644089 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-addd79b5-7ce7-4216-a1fb-ef2d5477739a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:33.673861 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3feaa00-39b1-4ff7-bdf3-6d282b15925a None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:33.697886 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:33.712878 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97cff0-18b8-42eb-b86e-f2abcc9be5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:33.713581 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97cff0-18b8-42eb-b86e-f2abcc9be5e8 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:33.713863 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97cff0-18b8-42eb-b86e-f2abcc9be5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:33.714137 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97cff0-18b8-42eb-b86e-f2abcc9be5e8 None admin] PATH_INFO: `/v3/services` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:33.714858 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97cff0-18b8-42eb-b86e-f2abcc9be5e8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:33.715170 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97cff0-18b8-42eb-b86e-f2abcc9be5e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:33.717344 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97cff0-18b8-42eb-b86e-f2abcc9be5e8 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:33.724427 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 477/965] 10.222.0.50 () {56 vars in 1122 bytes} [Fri May 27 03:10:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:33.725289 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 489/966] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:10:33.730082 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd79b5-7ce7-4216-a1fb-ef2d5477739a None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:33.734526 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97cff0-18b8-42eb-b86e-f2abcc9be5e8 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:33.761028 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:33.845450 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd79b5-7ce7-4216-a1fb-ef2d5477739a None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:33.855805 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97cff0-18b8-42eb-b86e-f2abcc9be5e8 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:33.886441 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb04218-186d-472e-b2a2-1c6407e516b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:33.887038 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb04218-186d-472e-b2a2-1c6407e516b7 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:33.887318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb04218-186d-472e-b2a2-1c6407e516b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:33.887596 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb04218-186d-472e-b2a2-1c6407e516b7 None admin] PATH_INFO: `/v3/services` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:33.888302 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb04218-186d-472e-b2a2-1c6407e516b7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:33.888607 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb04218-186d-472e-b2a2-1c6407e516b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:33.890120 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb04218-186d-472e-b2a2-1c6407e516b7 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:33.893463 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed60dbf1-ff34-4c1b-82e8-451089570306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:33.894242 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60dbf1-ff34-4c1b-82e8-451089570306 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:33.894520 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60dbf1-ff34-4c1b-82e8-451089570306 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:33.894795 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60dbf1-ff34-4c1b-82e8-451089570306 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:33.899889 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 490/967] 10.222.0.50 () {56 vars in 1122 bytes} [Fri May 27 03:10:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:33.899889 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb04218-186d-472e-b2a2-1c6407e516b7 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:33.907779 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed60dbf1-ff34-4c1b-82e8-451089570306 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:33.909763 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed60dbf1-ff34-4c1b-82e8-451089570306 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:33.925143 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-ed60dbf1-ff34-4c1b-82e8-451089570306 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:10:33.925143 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:33.925143 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:33.925143 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:33.925143 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:33.925143 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:33.925143 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:33.925143 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:33.925143 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:33.925143 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:33.925143 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:33.925143 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:33.925143 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:10:33.925143 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:10:33.925143 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:10:33.925143 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:10:33.926141 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:33.926141 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:33.926141 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:10:33.926141 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:33.926141 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:10:33.926141 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:33.926141 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:33.926141 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:33.926141 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:33.926141 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:33.926141 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:33.926141 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:33.926141 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:33.926141 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:33.926141 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:33.926141 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:33.926141 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:33.926141 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:33.927069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:33.927069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:33.927069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:10:33.927069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:10:33.927069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:10:33.927069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:10:33.927069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:10:33.927069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:10:33.927069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 03:10:33.927069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:33.927069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 478/968] 10.222.0.50 () {56 vars in 1117 bytes} [Fri May 27 03:10:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:33.927069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed60dbf1-ff34-4c1b-82e8-451089570306 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:33.978963 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:33.997745 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:34.007383 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb04218-186d-472e-b2a2-1c6407e516b7 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:34.031923 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed60dbf1-ff34-4c1b-82e8-451089570306 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:34.041673 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a5f57-c932-4c55-a5ae-fa47eea57500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:34.042343 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a5f57-c932-4c55-a5ae-fa47eea57500 None admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:34.042632 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a5f57-c932-4c55-a5ae-fa47eea57500 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:34.042904 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a5f57-c932-4c55-a5ae-fa47eea57500 None admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:34.043598 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a5f57-c932-4c55-a5ae-fa47eea57500 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:34.045087 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1a5f57-c932-4c55-a5ae-fa47eea57500 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:34.046525 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: WARNING keystone.server.flask.common [None req-7a1a5f57-c932-4c55-a5ae-fa47eea57500 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 27 03:10:34.059731 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.sql.core [None req-7a1a5f57-c932-4c55-a5ae-fa47eea57500 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 27 03:10:34.059731 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 27 03:10:34.059731 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [parameters: {'user_id': '20c17b92b41f40938d44cb8627e677e8', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 27 03:10:34.059731 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80409) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 27 03:10:34.072048 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da01be6-fdfa-4a38-bb83-6c5d8ed49f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:34.073044 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da01be6-fdfa-4a38-bb83-6c5d8ed49f3c None admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:34.073463 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da01be6-fdfa-4a38-bb83-6c5d8ed49f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:34.073866 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da01be6-fdfa-4a38-bb83-6c5d8ed49f3c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:34.074995 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da01be6-fdfa-4a38-bb83-6c5d8ed49f3c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:34.076017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-7a1a5f57-c932-4c55-a5ae-fa47eea57500 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 27 03:10:34.076017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:34.076017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 03:10:34.076017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 03:10:34.076017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 03:10:34.076017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 03:10:34.076017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 03:10:34.076017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application result = self._query(query) May 27 03:10:34.076017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 03:10:34.076017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application conn.query(q) May 27 03:10:34.076017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 03:10:34.076017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 03:10:34.076017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 03:10:34.076017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application result.read() May 27 03:10:34.076017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 03:10:34.076843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 03:10:34.076843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 03:10:34.076843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 03:10:34.076843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 03:10:34.076843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 03:10:34.076843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 03:10:34.076843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 03:10:34.076843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 27 03:10:34.076843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:34.076843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 27 03:10:34.076843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:34.076843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:34.076843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 27 03:10:34.076843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 27 03:10:34.076843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 27 03:10:34.076843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 27 03:10:34.076843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 03:10:34.076843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application next(self.gen) May 27 03:10:34.077841 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 27 03:10:34.077841 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application yield resource May 27 03:10:34.077841 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 03:10:34.077841 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application next(self.gen) May 27 03:10:34.077841 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 27 03:10:34.077841 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application self.session.rollback() May 27 03:10:34.077841 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 27 03:10:34.077841 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application self.force_reraise() May 27 03:10:34.077841 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 27 03:10:34.077841 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise self.value May 27 03:10:34.077841 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 27 03:10:34.077841 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 27 03:10:34.077841 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 27 03:10:34.077841 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application session.commit() May 27 03:10:34.077841 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 27 03:10:34.077841 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 27 03:10:34.077841 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 27 03:10:34.079381 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application self._prepare_impl() May 27 03:10:34.079381 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 27 03:10:34.079381 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application self.session.flush() May 27 03:10:34.079381 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 27 03:10:34.079381 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application self._flush(objects) May 27 03:10:34.079381 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 27 03:10:34.079381 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 27 03:10:34.079381 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 27 03:10:34.079381 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application compat.raise_( May 27 03:10:34.079381 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 03:10:34.079381 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception May 27 03:10:34.079381 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 27 03:10:34.079381 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application flush_context.execute() May 27 03:10:34.079381 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 27 03:10:34.079381 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rec.execute(self) May 27 03:10:34.079381 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 27 03:10:34.079381 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 27 03:10:34.080849 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 27 03:10:34.080849 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application _emit_insert_statements( May 27 03:10:34.080849 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 27 03:10:34.080849 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application result = connection._execute_20( May 27 03:10:34.080849 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 27 03:10:34.080849 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 27 03:10:34.080849 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 27 03:10:34.080849 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 27 03:10:34.080849 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 27 03:10:34.080849 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ret = self._execute_context( May 27 03:10:34.080849 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 27 03:10:34.080849 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 27 03:10:34.080849 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 27 03:10:34.080849 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 27 03:10:34.080849 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 03:10:34.080849 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception May 27 03:10:34.084069 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 03:10:34.084069 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 03:10:34.084069 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 03:10:34.084069 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 03:10:34.084069 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 03:10:34.084069 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application result = self._query(query) May 27 03:10:34.084069 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 03:10:34.084069 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application conn.query(q) May 27 03:10:34.084069 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 03:10:34.084069 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 03:10:34.084069 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 03:10:34.084069 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application result.read() May 27 03:10:34.084069 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 03:10:34.084069 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 03:10:34.084069 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 03:10:34.084069 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 03:10:34.085124 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 03:10:34.085124 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 03:10:34.085124 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 03:10:34.085124 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 03:10:34.085124 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 27 03:10:34.085124 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 27 03:10:34.085124 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [parameters: {'user_id': '20c17b92b41f40938d44cb8627e677e8', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 27 03:10:34.085124 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 27 03:10:34.085124 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:34.085124 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:10:34.085124 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:34.085124 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:34.085124 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:34.085124 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:34.085124 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:34.086112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:34.086112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:34.086112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:34.086112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:34.086112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:34.086112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:34.086112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:34.086112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 27 03:10:34.086112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 27 03:10:34.086112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:34.086112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:34.086112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:10:34.086112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:34.086112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:10:34.086112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:34.086112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 27 03:10:34.086112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 27 03:10:34.087469 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 27 03:10:34.087469 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 27 03:10:34.087469 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 27 03:10:34.087469 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 27 03:10:34.087469 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 27 03:10:34.087469 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:34.088151 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da01be6-fdfa-4a38-bb83-6c5d8ed49f3c None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:34.088283 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 491/969] 10.222.0.50 () {60 vars in 1131 bytes} [Fri May 27 03:10:33 2022] POST /identity/v3/users => generated 160 bytes in 181 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:34.088283 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a5f57-c932-4c55-a5ae-fa47eea57500 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:34.104324 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 479/970] 10.222.0.50 () {60 vars in 1144 bytes} [Fri May 27 03:10:33 2022] POST /identity/v3/endpoints => generated 349 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:10:34.105593 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f6f48-ac48-4fe4-893b-67c553aeb424 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:34.105882 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f6f48-ac48-4fe4-893b-67c553aeb424 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:34.106155 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f6f48-ac48-4fe4-893b-67c553aeb424 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:34.112520 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 480/971] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:34.112520 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f6f48-ac48-4fe4-893b-67c553aeb424 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:34.202794 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a5f57-c932-4c55-a5ae-fa47eea57500 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:34.233086 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:34.240616 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c2263-3399-4e80-8186-8aa4edb3bb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:34.240891 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f6f48-ac48-4fe4-893b-67c553aeb424 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:34.241470 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c2263-3399-4e80-8186-8aa4edb3bb9c None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:34.241727 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c2263-3399-4e80-8186-8aa4edb3bb9c None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:34.242001 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c2263-3399-4e80-8186-8aa4edb3bb9c None admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:34.242969 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75c2263-3399-4e80-8186-8aa4edb3bb9c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:34.243281 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75c2263-3399-4e80-8186-8aa4edb3bb9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:34.245007 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75c2263-3399-4e80-8186-8aa4edb3bb9c None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:34.273246 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 492/972] 10.222.0.50 () {56 vars in 1149 bytes} [Fri May 27 03:10:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:34.273246 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cecfe1-af44-4b9c-86d9-f0297f3947f5 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:34.273246 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cecfe1-af44-4b9c-86d9-f0297f3947f5 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:34.273246 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cecfe1-af44-4b9c-86d9-f0297f3947f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:34.289897 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a980e-4852-4288-9dd8-f897ac6a2ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:34.290707 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a980e-4852-4288-9dd8-f897ac6a2ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:34.290980 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a980e-4852-4288-9dd8-f897ac6a2ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:34.291246 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a980e-4852-4288-9dd8-f897ac6a2ab7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:34.291889 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-59cecfe1-af44-4b9c-86d9-f0297f3947f5 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:34.295711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-59cecfe1-af44-4b9c-86d9-f0297f3947f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:34.300684 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5a980e-4852-4288-9dd8-f897ac6a2ab7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:34.302179 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5a980e-4852-4288-9dd8-f897ac6a2ab7 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:34.302397 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:34.317651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-1f5a980e-4852-4288-9dd8-f897ac6a2ab7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 27 03:10:34.317651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:34.317651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:34.317651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:34.317651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:34.317651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:34.317651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:34.317651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:34.317651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:34.317651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:34.317651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:34.317651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:34.317651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:10:34.317651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:10:34.317651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:10:34.317651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:10:34.318829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:34.318829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:34.318829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:10:34.318829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:34.318829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:10:34.318829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:34.318829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:34.318829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:34.318829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:34.318829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:34.318829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:34.318829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:34.318829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:34.318829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:34.318829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:34.318829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:34.318829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:34.318829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:34.319793 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:34.319793 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:34.319793 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:10:34.319793 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:10:34.319793 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:10:34.319793 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:10:34.319793 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:10:34.319793 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:10:34.319793 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 27 03:10:34.319793 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:34.319793 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 481/973] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:10:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:34.320391 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a980e-4852-4288-9dd8-f897ac6a2ab7 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:34.373582 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 493/974] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:10:34.374688 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cecfe1-af44-4b9c-86d9-f0297f3947f5 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:34.439037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a980e-4852-4288-9dd8-f897ac6a2ab7 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:34.473787 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6445125-9bb7-42e7-99d9-04dde9a14c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:34.474353 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6445125-9bb7-42e7-99d9-04dde9a14c01 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:34.474626 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6445125-9bb7-42e7-99d9-04dde9a14c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:34.474894 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6445125-9bb7-42e7-99d9-04dde9a14c01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:34.494031 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cecfe1-af44-4b9c-86d9-f0297f3947f5 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:34.504828 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:34.530881 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39558b29-bd92-4f1c-97c5-1c41993cb175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:34.531731 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39558b29-bd92-4f1c-97c5-1c41993cb175 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:34.532104 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39558b29-bd92-4f1c-97c5-1c41993cb175 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:34.532871 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39558b29-bd92-4f1c-97c5-1c41993cb175 None admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:34.534818 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39558b29-bd92-4f1c-97c5-1c41993cb175 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:34.535214 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39558b29-bd92-4f1c-97c5-1c41993cb175 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:34.537549 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39558b29-bd92-4f1c-97c5-1c41993cb175 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:34.548432 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6445125-9bb7-42e7-99d9-04dde9a14c01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:34.549925 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6445125-9bb7-42e7-99d9-04dde9a14c01 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:34.566433 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:34.578310 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 494/975] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 03:10:34 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:34.579089 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39558b29-bd92-4f1c-97c5-1c41993cb175 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:34.702909 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 482/976] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:10:34 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:10:34.710175 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6445125-9bb7-42e7-99d9-04dde9a14c01 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:34.712961 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39558b29-bd92-4f1c-97c5-1c41993cb175 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:34.752210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f238ba13-f0c8-4c12-a77e-84b91c13bb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:34.753156 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238ba13-f0c8-4c12-a77e-84b91c13bb8c None admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:34.753652 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238ba13-f0c8-4c12-a77e-84b91c13bb8c None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:34.754030 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238ba13-f0c8-4c12-a77e-84b91c13bb8c None admin] PATH_INFO: `/v3/services` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:34.754882 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f238ba13-f0c8-4c12-a77e-84b91c13bb8c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:34.756550 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f238ba13-f0c8-4c12-a77e-84b91c13bb8c None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:34.775096 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:34.775342 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 495/977] 10.222.0.50 () {60 vars in 1141 bytes} [Fri May 27 03:10:34 2022] POST /identity/v3/services => generated 242 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:10:34.836311 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6445125-9bb7-42e7-99d9-04dde9a14c01 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:34.886376 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e59abe8-a29a-42aa-bc76-0dd520d4709b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:34.887205 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59abe8-a29a-42aa-bc76-0dd520d4709b None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:34.887458 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59abe8-a29a-42aa-bc76-0dd520d4709b None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:34.887729 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59abe8-a29a-42aa-bc76-0dd520d4709b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:34.900632 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e59abe8-a29a-42aa-bc76-0dd520d4709b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:34.902264 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e59abe8-a29a-42aa-bc76-0dd520d4709b None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:34.922915 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-7e59abe8-a29a-42aa-bc76-0dd520d4709b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:34.922915 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:34.922915 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:34.922915 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:34.922915 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:34.922915 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:34.922915 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:34.922915 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:34.922915 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:34.922915 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:34.922915 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:34.922915 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:34.922915 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:34.922915 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:34.922915 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:34.923888 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:34.923888 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:34.923888 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:34.923888 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:34.923888 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:34.923888 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:34.923888 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:34.923888 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:34.923888 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:34.923888 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:34.923888 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:34.923888 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:34.923888 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:34.923888 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:34.923888 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:34.923888 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:34.923888 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:34.925326 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:34.925326 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:34.925326 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:34.925326 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:34.925326 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 483/978] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:34.930861 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f238ba13-f0c8-4c12-a77e-84b91c13bb8c None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:35.000205 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:35.047737 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f238ba13-f0c8-4c12-a77e-84b91c13bb8c None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:35.085031 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2001b5-eee1-4acb-a5ba-aa520efc96b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:35.085703 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2001b5-eee1-4acb-a5ba-aa520efc96b7 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:35.085983 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2001b5-eee1-4acb-a5ba-aa520efc96b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:35.086255 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2001b5-eee1-4acb-a5ba-aa520efc96b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:35.086968 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2001b5-eee1-4acb-a5ba-aa520efc96b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:35.087272 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2001b5-eee1-4acb-a5ba-aa520efc96b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:35.088639 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2001b5-eee1-4acb-a5ba-aa520efc96b7 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:35.101329 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 496/979] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:10:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:35.106930 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e59abe8-a29a-42aa-bc76-0dd520d4709b None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:35.211188 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e59abe8-a29a-42aa-bc76-0dd520d4709b None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:35.219086 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:35.248600 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a583f5ef-83bf-4644-84f1-35a7e434843f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:35.249494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a583f5ef-83bf-4644-84f1-35a7e434843f None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:35.249771 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a583f5ef-83bf-4644-84f1-35a7e434843f None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:35.250040 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a583f5ef-83bf-4644-84f1-35a7e434843f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:35.257285 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a583f5ef-83bf-4644-84f1-35a7e434843f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:35.258741 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a583f5ef-83bf-4644-84f1-35a7e434843f None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:35.272913 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-a583f5ef-83bf-4644-84f1-35a7e434843f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:35.272913 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:35.272913 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:35.272913 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:35.272913 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:35.272913 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:35.272913 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:35.272913 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:35.272913 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:35.272913 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:35.272913 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:35.272913 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:35.272913 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:10:35.272913 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:10:35.272913 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:10:35.273859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:10:35.273859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:35.273859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:35.273859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:35.273859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:35.273859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:35.273859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:35.273859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:35.273859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:35.273859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:35.273859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:35.273859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:35.273859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:35.273859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:35.273859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:35.273859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:35.273859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:35.274760 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:10:35.274760 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:10:35.274760 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:10:35.274760 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:10:35.274760 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:10:35.274760 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:10:35.274760 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:35.274760 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:35.279021 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 484/980] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:10:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:35.280479 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2001b5-eee1-4acb-a5ba-aa520efc96b7 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:35.395165 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2001b5-eee1-4acb-a5ba-aa520efc96b7 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:35.414905 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:35.433123 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c107be6c-7fbf-4318-a688-01d40b693419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:35.433794 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107be6c-7fbf-4318-a688-01d40b693419 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:35.434078 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107be6c-7fbf-4318-a688-01d40b693419 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:35.434352 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107be6c-7fbf-4318-a688-01d40b693419 None admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:35.435140 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c107be6c-7fbf-4318-a688-01d40b693419 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:35.435448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c107be6c-7fbf-4318-a688-01d40b693419 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:35.436902 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c107be6c-7fbf-4318-a688-01d40b693419 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:35.454299 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 497/981] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 03:10:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:35.455919 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a583f5ef-83bf-4644-84f1-35a7e434843f None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:35.563266 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a583f5ef-83bf-4644-84f1-35a7e434843f None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:35.598981 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d140d-fa4a-44ad-96c0-ecdc19cd1b6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:35.599641 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d140d-fa4a-44ad-96c0-ecdc19cd1b6c None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:35.599920 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d140d-fa4a-44ad-96c0-ecdc19cd1b6c None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:35.600177 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d140d-fa4a-44ad-96c0-ecdc19cd1b6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:35.601076 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d140d-fa4a-44ad-96c0-ecdc19cd1b6c None admin] RBAC: Adding query filter params (role.id=8c199022919d4ef2b185859d8e1dc9a9, scope.project.id=f8398941252346468483ea0c863122f4, user.id=86a18e313e4944748aef58d54bfe7b13) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:35.601390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d140d-fa4a-44ad-96c0-ecdc19cd1b6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:35.602829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d140d-fa4a-44ad-96c0-ecdc19cd1b6c None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:35.609394 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 485/982] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:10:35 2022] GET /identity/v3/role_assignments?user.id=86a18e313e4944748aef58d54bfe7b13&scope.project.id=f8398941252346468483ea0c863122f4&role.id=8c199022919d4ef2b185859d8e1dc9a9 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:36.393748 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10bacd2-7146-4f07-93fa-59b95bf15262 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:36.394294 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10bacd2-7146-4f07-93fa-59b95bf15262 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:36.394294 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10bacd2-7146-4f07-93fa-59b95bf15262 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:36.395652 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 498/983] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:36.401308 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a4f29-4447-4abd-af83-809125004ffc None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:36.401728 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a4f29-4447-4abd-af83-809125004ffc None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:36.401819 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25a4f29-4447-4abd-af83-809125004ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:36.419671 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-d25a4f29-4447-4abd-af83-809125004ffc None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:36.420863 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-d25a4f29-4447-4abd-af83-809125004ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:36.445926 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:36.492323 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 486/984] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:36 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:10:36.514953 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10bacd2-7146-4f07-93fa-59b95bf15262 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:36.623951 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10bacd2-7146-4f07-93fa-59b95bf15262 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:36.658183 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3cfe7c-3f8d-4d66-9cf9-0e396e2de973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:36.659462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3cfe7c-3f8d-4d66-9cf9-0e396e2de973 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:36.659777 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3cfe7c-3f8d-4d66-9cf9-0e396e2de973 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:36.660051 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3cfe7c-3f8d-4d66-9cf9-0e396e2de973 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:36.660767 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3cfe7c-3f8d-4d66-9cf9-0e396e2de973 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:36.662183 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3cfe7c-3f8d-4d66-9cf9-0e396e2de973 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:36.674251 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-1c3cfe7c-3f8d-4d66-9cf9-0e396e2de973 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:10:36.674251 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:36.674251 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:10:36.674251 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:10:36.674251 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:10:36.674251 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:10:36.674251 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:10:36.674251 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:36.674251 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:10:36.674251 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:36.674251 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:10:36.674251 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:36.674251 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:36.674251 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:36.674251 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:36.674251 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:36.674251 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:36.675417 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:36.675417 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:36.675417 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:36.675417 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:36.675417 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:36.675417 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:36.675417 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:10:36.675417 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:10:36.675417 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:10:36.675417 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:10:36.675417 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:36.675417 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:36.675417 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:36.675417 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:36.675417 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:36.675417 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:36.681448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:36.681448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:36.681448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:36.681448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:36.681448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:36.681448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:36.681448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:36.681448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:36.681448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:36.681448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:36.681448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:10:36.681448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:36.681448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:10:36.681448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:36.681448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 499/985] 10.222.0.50 () {56 vars in 1126 bytes} [Fri May 27 03:10:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:36.683915 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a4f29-4447-4abd-af83-809125004ffc None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:36.795125 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25a4f29-4447-4abd-af83-809125004ffc None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:36.831034 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4076d3be-9cc4-4932-95c0-422b50afa2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:36.831723 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076d3be-9cc4-4932-95c0-422b50afa2b0 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:36.832005 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076d3be-9cc4-4932-95c0-422b50afa2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:36.832283 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4076d3be-9cc4-4932-95c0-422b50afa2b0 None admin] PATH_INFO: `/v3/services` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:36.833031 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4076d3be-9cc4-4932-95c0-422b50afa2b0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:36.833336 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4076d3be-9cc4-4932-95c0-422b50afa2b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:36.834702 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4076d3be-9cc4-4932-95c0-422b50afa2b0 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:36.844672 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 487/986] 10.222.0.50 () {56 vars in 1122 bytes} [Fri May 27 03:10:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:36.845728 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3cfe7c-3f8d-4d66-9cf9-0e396e2de973 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:36.896033 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:36.959306 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3cfe7c-3f8d-4d66-9cf9-0e396e2de973 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:36.996569 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62c6a55-e246-41ce-8e05-20adf4a3abc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:36.997551 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c6a55-e246-41ce-8e05-20adf4a3abc3 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:36.998069 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c6a55-e246-41ce-8e05-20adf4a3abc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:36.998482 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62c6a55-e246-41ce-8e05-20adf4a3abc3 None admin] PATH_INFO: `/v3/services` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:36.999317 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62c6a55-e246-41ce-8e05-20adf4a3abc3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:36.999750 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62c6a55-e246-41ce-8e05-20adf4a3abc3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:37.001304 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62c6a55-e246-41ce-8e05-20adf4a3abc3 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:37.011661 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 500/987] 10.222.0.50 () {56 vars in 1122 bytes} [Fri May 27 03:10:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:37.013026 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4076d3be-9cc4-4932-95c0-422b50afa2b0 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:37.040042 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:37.056169 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270ae3e-a2bc-4961-863d-d5e750beb384 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:37.056455 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270ae3e-a2bc-4961-863d-d5e750beb384 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:37.057075 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270ae3e-a2bc-4961-863d-d5e750beb384 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:37.058466 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 501/988] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:10:37.063176 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8efa7a-6e63-49ee-ad20-674dad86de69 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:37.063467 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8efa7a-6e63-49ee-ad20-674dad86de69 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:37.063738 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8efa7a-6e63-49ee-ad20-674dad86de69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:37.081826 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-6f8efa7a-6e63-49ee-ad20-674dad86de69 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:37.082995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-6f8efa7a-6e63-49ee-ad20-674dad86de69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:37.121452 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4076d3be-9cc4-4932-95c0-422b50afa2b0 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:37.155742 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c94e54-c211-4eb6-a1ea-723adc2f2d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:37.156317 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c94e54-c211-4eb6-a1ea-723adc2f2d55 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:37.160987 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c94e54-c211-4eb6-a1ea-723adc2f2d55 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:37.161141 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 502/989] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:37 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:10:37.161227 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c94e54-c211-4eb6-a1ea-723adc2f2d55 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:37.161227 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c94e54-c211-4eb6-a1ea-723adc2f2d55 None admin] RBAC: Adding query filter params (interface=admin, service_id=2a7fc41176514aa1b8623624394cb650, region_id=RegionOne) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:37.161227 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c94e54-c211-4eb6-a1ea-723adc2f2d55 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:37.161227 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c94e54-c211-4eb6-a1ea-723adc2f2d55 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:37.168995 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 488/990] 10.222.0.50 () {56 vars in 1249 bytes} [Fri May 27 03:10:37 2022] GET /identity/v3/endpoints?service_id=2a7fc41176514aa1b8623624394cb650&interface=admin®ion_id=RegionOne => generated 193 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:37.171040 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8efa7a-6e63-49ee-ad20-674dad86de69 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:37.181424 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c94e54-c211-4eb6-a1ea-723adc2f2d55 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:37.211777 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:37.272538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:37.285277 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8efa7a-6e63-49ee-ad20-674dad86de69 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:37.285998 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c94e54-c211-4eb6-a1ea-723adc2f2d55 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:37.324980 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef08c6-84d9-4fd2-86b9-2ccaefef834a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:37.325783 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef08c6-84d9-4fd2-86b9-2ccaefef834a None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:37.326062 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef08c6-84d9-4fd2-86b9-2ccaefef834a None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:37.326332 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef08c6-84d9-4fd2-86b9-2ccaefef834a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:37.326980 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef08c6-84d9-4fd2-86b9-2ccaefef834a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:37.327597 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6630b925-ea2f-40c9-808c-c10e9b7346ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:37.328330 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630b925-ea2f-40c9-808c-c10e9b7346ff None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:37.328423 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef08c6-84d9-4fd2-86b9-2ccaefef834a None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:37.328700 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630b925-ea2f-40c9-808c-c10e9b7346ff None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:37.329067 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6630b925-ea2f-40c9-808c-c10e9b7346ff None admin] PATH_INFO: `/v3/services` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:37.329808 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6630b925-ea2f-40c9-808c-c10e9b7346ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:37.331501 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6630b925-ea2f-40c9-808c-c10e9b7346ff None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:37.338296 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 503/991] 10.222.0.50 () {56 vars in 1087 bytes} [Fri May 27 03:10:37 2022] GET /identity/v3/services => generated 1727 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:10:37.340616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-daef08c6-84d9-4fd2-86b9-2ccaefef834a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 27 03:10:37.340616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:37.340616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:10:37.340616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:10:37.340616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:10:37.340616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:10:37.340616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:10:37.340616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:37.340616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 03:10:37.340616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:37.340616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:10:37.340616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:37.340616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:37.340616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:37.340616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:37.340616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:37.340616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:37.341754 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:37.341754 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:37.341754 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:37.341754 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:37.341754 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:37.341754 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:37.341754 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:10:37.341754 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:10:37.341754 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:10:37.341754 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:10:37.341754 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:37.341754 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:37.341754 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:37.341754 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:37.341754 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:37.341754 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:37.342618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:37.342618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:37.342618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:37.342618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:37.342618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:37.342618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:37.342618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:37.342618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:37.342618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:37.342618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:37.342618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:10:37.342618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:37.342618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 03:10:37.342618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:37.342618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 489/992] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:10:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:10:37.351472 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6630b925-ea2f-40c9-808c-c10e9b7346ff None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:37.464289 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:37.468858 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6630b925-ea2f-40c9-808c-c10e9b7346ff None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:37.509257 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e38d377-a5d2-439f-9f26-5f7daeba0370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:37.509932 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e38d377-a5d2-439f-9f26-5f7daeba0370 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:37.510203 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e38d377-a5d2-439f-9f26-5f7daeba0370 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:37.510472 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e38d377-a5d2-439f-9f26-5f7daeba0370 None admin] PATH_INFO: `/v3/services` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:37.511178 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e38d377-a5d2-439f-9f26-5f7daeba0370 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:37.511477 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e38d377-a5d2-439f-9f26-5f7daeba0370 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:37.512883 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e38d377-a5d2-439f-9f26-5f7daeba0370 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:37.519134 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 504/993] 10.222.0.50 () {56 vars in 1112 bytes} [Fri May 27 03:10:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:37.524765 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef08c6-84d9-4fd2-86b9-2ccaefef834a None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:37.632533 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef08c6-84d9-4fd2-86b9-2ccaefef834a None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:37.670966 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d5137-4966-432c-8ef7-1ef91eea1b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:37.671998 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d5137-4966-432c-8ef7-1ef91eea1b5b None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:37.672478 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d5137-4966-432c-8ef7-1ef91eea1b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:37.672995 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d5137-4966-432c-8ef7-1ef91eea1b5b None admin] PATH_INFO: `/v3/services` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:37.673963 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0d5137-4966-432c-8ef7-1ef91eea1b5b None admin] RBAC: Adding query filter params (type=compute) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:37.675231 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0d5137-4966-432c-8ef7-1ef91eea1b5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:37.676885 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0d5137-4966-432c-8ef7-1ef91eea1b5b None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:37.683207 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 490/994] 10.222.0.50 () {56 vars in 1112 bytes} [Fri May 27 03:10:37 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:37.688854 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e38d377-a5d2-439f-9f26-5f7daeba0370 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:37.804131 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e38d377-a5d2-439f-9f26-5f7daeba0370 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:37.840034 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0099b06-063c-40ba-b5a1-30e6049e2170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:37.840695 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0099b06-063c-40ba-b5a1-30e6049e2170 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:37.840985 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0099b06-063c-40ba-b5a1-30e6049e2170 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:37.841258 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0099b06-063c-40ba-b5a1-30e6049e2170 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:37.842036 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0099b06-063c-40ba-b5a1-30e6049e2170 None admin] RBAC: Adding query filter params (interface=public, service_id=478d5890035e43b3aa13cb138d5da853, region_id=RegionOne) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:37.842337 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0099b06-063c-40ba-b5a1-30e6049e2170 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:37.843713 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0099b06-063c-40ba-b5a1-30e6049e2170 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:37.849438 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 505/995] 10.222.0.50 () {56 vars in 1251 bytes} [Fri May 27 03:10:37 2022] GET /identity/v3/endpoints?service_id=478d5890035e43b3aa13cb138d5da853&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:37.855054 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d5137-4966-432c-8ef7-1ef91eea1b5b None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:37.877166 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:37.906942 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79cfa8-2343-44eb-98d9-2fa4746e7e4a None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:37.907232 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79cfa8-2343-44eb-98d9-2fa4746e7e4a None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:37.907492 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c79cfa8-2343-44eb-98d9-2fa4746e7e4a None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:37.909614 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 506/996] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:10:37.914468 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6218e5-f69f-4443-9362-a0d0772b54e4 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:37.914754 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6218e5-f69f-4443-9362-a0d0772b54e4 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:37.915025 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6218e5-f69f-4443-9362-a0d0772b54e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:37.932372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-8f6218e5-f69f-4443-9362-a0d0772b54e4 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:37.933589 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-8f6218e5-f69f-4443-9362-a0d0772b54e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:37.963705 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:37.974554 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d5137-4966-432c-8ef7-1ef91eea1b5b None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:38.012218 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b5a065-0541-4e55-a06a-b14942f7e1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:38.013287 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5a065-0541-4e55-a06a-b14942f7e1e1 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:38.013443 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 507/997] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:37 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:10:38.013603 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5a065-0541-4e55-a06a-b14942f7e1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:38.013603 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5a065-0541-4e55-a06a-b14942f7e1e1 None admin] PATH_INFO: `/v3/services` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:38.014191 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b5a065-0541-4e55-a06a-b14942f7e1e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:38.015668 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b5a065-0541-4e55-a06a-b14942f7e1e1 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:38.025363 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 491/998] 10.222.0.50 () {56 vars in 1087 bytes} [Fri May 27 03:10:37 2022] GET /identity/v3/services => generated 1727 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:10:38.035798 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6218e5-f69f-4443-9362-a0d0772b54e4 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:38.150464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6218e5-f69f-4443-9362-a0d0772b54e4 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:38.192121 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498ad21-37e2-4c3f-9fa0-c66c9999399a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:38.192939 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498ad21-37e2-4c3f-9fa0-c66c9999399a None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:38.193246 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498ad21-37e2-4c3f-9fa0-c66c9999399a None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:38.193521 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498ad21-37e2-4c3f-9fa0-c66c9999399a None admin] PATH_INFO: `/v3/services/image` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:38.194180 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6498ad21-37e2-4c3f-9fa0-c66c9999399a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:38.195641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6498ad21-37e2-4c3f-9fa0-c66c9999399a None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:38.208222 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-6498ad21-37e2-4c3f-9fa0-c66c9999399a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 27 03:10:38.208222 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:38.208222 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:10:38.208222 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:10:38.208222 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:10:38.208222 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:10:38.208222 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:10:38.208222 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:38.208222 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 03:10:38.208222 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:38.208222 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:10:38.208222 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:38.208222 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:38.208222 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:38.208222 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:38.208222 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:38.208222 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:38.209372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:38.209372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:38.209372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:38.209372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:38.209372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:38.209372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:38.209372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:10:38.209372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:10:38.209372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:10:38.209372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:10:38.209372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:38.209372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:38.209372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:38.209372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:38.209372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:38.209372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:38.215675 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:38.215675 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:38.215675 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:38.215675 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:38.215675 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:38.215675 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:38.215675 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:38.215675 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:38.215675 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:38.215675 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:38.215675 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:10:38.215675 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:38.215675 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 03:10:38.215675 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:38.215675 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 508/999] 10.222.0.50 () {56 vars in 1105 bytes} [Fri May 27 03:10:38 2022] GET /identity/v3/services/image => generated 86 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:10:38.216465 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b5a065-0541-4e55-a06a-b14942f7e1e1 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:38.299595 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:38.321768 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b5a065-0541-4e55-a06a-b14942f7e1e1 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:38.357229 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aaee82-b124-46ec-a2f7-936a99d582d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:38.357892 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aaee82-b124-46ec-a2f7-936a99d582d8 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:38.358159 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aaee82-b124-46ec-a2f7-936a99d582d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:38.358432 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aaee82-b124-46ec-a2f7-936a99d582d8 None admin] PATH_INFO: `/v3/services` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:38.359197 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aaee82-b124-46ec-a2f7-936a99d582d8 None admin] RBAC: Adding query filter params (name=image) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:38.359520 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aaee82-b124-46ec-a2f7-936a99d582d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:38.360881 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aaee82-b124-46ec-a2f7-936a99d582d8 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:38.366220 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 492/1000] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:38 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:38.371923 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498ad21-37e2-4c3f-9fa0-c66c9999399a None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:38.399003 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:38.480570 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6498ad21-37e2-4c3f-9fa0-c66c9999399a None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:38.514217 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946fe5f-004d-4925-a3da-e16434feb5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:38.514888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946fe5f-004d-4925-a3da-e16434feb5bc None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:38.515166 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946fe5f-004d-4925-a3da-e16434feb5bc None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:38.515436 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946fe5f-004d-4925-a3da-e16434feb5bc None admin] PATH_INFO: `/v3/services` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:38.516129 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d946fe5f-004d-4925-a3da-e16434feb5bc None admin] RBAC: Adding query filter params (type=image) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:38.516435 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d946fe5f-004d-4925-a3da-e16434feb5bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:38.517866 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d946fe5f-004d-4925-a3da-e16434feb5bc None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:38.522734 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 509/1001] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:38 2022] GET /identity/v3/services?type=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:39.644172 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da067510-dcae-44ce-9b1c-e470c3f41b2c None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:39.644707 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da067510-dcae-44ce-9b1c-e470c3f41b2c None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:39.644707 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da067510-dcae-44ce-9b1c-e470c3f41b2c None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:39.646386 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 493/1002] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:39.652193 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bce37b-cb57-43fa-a6dc-e53f03baa7e5 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:39.652753 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bce37b-cb57-43fa-a6dc-e53f03baa7e5 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:39.652753 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bce37b-cb57-43fa-a6dc-e53f03baa7e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:39.670293 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-e4bce37b-cb57-43fa-a6dc-e53f03baa7e5 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:39.671481 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-e4bce37b-cb57-43fa-a6dc-e53f03baa7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:39.742265 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 510/1003] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:39 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:10:39.771397 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da067510-dcae-44ce-9b1c-e470c3f41b2c None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:39.884965 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da067510-dcae-44ce-9b1c-e470c3f41b2c None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:39.919055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f2c0c0-db3a-4875-8c34-092fb1f15a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:39.919789 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f2c0c0-db3a-4875-8c34-092fb1f15a0e None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:39.920065 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f2c0c0-db3a-4875-8c34-092fb1f15a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:39.920313 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f2c0c0-db3a-4875-8c34-092fb1f15a0e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:39.920992 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f2c0c0-db3a-4875-8c34-092fb1f15a0e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:39.922374 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f2c0c0-db3a-4875-8c34-092fb1f15a0e None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:39.934587 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-41f2c0c0-db3a-4875-8c34-092fb1f15a0e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:10:39.934587 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:39.934587 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:10:39.934587 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:10:39.934587 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:10:39.934587 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:10:39.934587 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:10:39.934587 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:39.934587 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:10:39.934587 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:39.934587 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:10:39.934587 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:39.934587 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:39.934587 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:39.934587 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:39.934587 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:39.934587 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:39.935559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:39.935559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:39.935559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:39.935559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:39.935559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:39.935559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:39.935559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:10:39.935559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:10:39.935559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:10:39.935559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:10:39.935559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:39.935559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:39.935559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:39.935559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:39.935559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:39.935559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:39.936404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:39.936404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:39.936404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:39.936404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:39.936404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:39.936404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:39.936404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:39.936404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:39.936404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:39.936404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:39.936404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:10:39.936404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:39.936404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 03:10:39.936404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:39.937990 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 494/1004] 10.222.0.50 () {56 vars in 1126 bytes} [Fri May 27 03:10:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:39.942166 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bce37b-cb57-43fa-a6dc-e53f03baa7e5 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:39.960760 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:40.044487 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bce37b-cb57-43fa-a6dc-e53f03baa7e5 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:40.088784 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d76a26-e66d-4fe4-b59e-e6086e02acae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:40.089499 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d76a26-e66d-4fe4-b59e-e6086e02acae None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:40.089778 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d76a26-e66d-4fe4-b59e-e6086e02acae None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:40.090056 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d76a26-e66d-4fe4-b59e-e6086e02acae None admin] PATH_INFO: `/v3/services` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:40.090764 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d76a26-e66d-4fe4-b59e-e6086e02acae None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:40.091071 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d76a26-e66d-4fe4-b59e-e6086e02acae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:40.092459 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d76a26-e66d-4fe4-b59e-e6086e02acae None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:40.098203 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 511/1005] 10.222.0.50 () {56 vars in 1122 bytes} [Fri May 27 03:10:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:40.103987 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f2c0c0-db3a-4875-8c34-092fb1f15a0e None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:40.116125 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:40.207023 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f2c0c0-db3a-4875-8c34-092fb1f15a0e None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:40.241300 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f650628-108f-4a9d-8e16-af0fd9457d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:40.241971 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f650628-108f-4a9d-8e16-af0fd9457d12 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:40.242254 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f650628-108f-4a9d-8e16-af0fd9457d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:40.242528 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f650628-108f-4a9d-8e16-af0fd9457d12 None admin] PATH_INFO: `/v3/services` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:40.243237 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f650628-108f-4a9d-8e16-af0fd9457d12 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:40.243540 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f650628-108f-4a9d-8e16-af0fd9457d12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:40.244921 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f650628-108f-4a9d-8e16-af0fd9457d12 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:40.250347 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 495/1006] 10.222.0.50 () {56 vars in 1122 bytes} [Fri May 27 03:10:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:40.254538 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90185956-5cf4-4c7b-aee6-3c89acaede91 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:40.254780 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90185956-5cf4-4c7b-aee6-3c89acaede91 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:40.256220 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90185956-5cf4-4c7b-aee6-3c89acaede91 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:40.256333 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f650628-108f-4a9d-8e16-af0fd9457d12 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:40.256432 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 512/1007] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:40.261515 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ad73e-c117-41f3-b946-2c45c05173c6 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:40.261794 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ad73e-c117-41f3-b946-2c45c05173c6 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:40.262062 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ad73e-c117-41f3-b946-2c45c05173c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:40.278814 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-683ad73e-c117-41f3-b946-2c45c05173c6 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:40.279977 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-683ad73e-c117-41f3-b946-2c45c05173c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:40.336443 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:40.369302 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f650628-108f-4a9d-8e16-af0fd9457d12 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:40.373256 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 513/1008] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:40 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:10:40.383385 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ad73e-c117-41f3-b946-2c45c05173c6 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:40.395735 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:40.407180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876e3354-e1bd-46ac-ad69-27f4f4236aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:40.407855 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e3354-e1bd-46ac-ad69-27f4f4236aa4 None admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:40.408140 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e3354-e1bd-46ac-ad69-27f4f4236aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:40.408926 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876e3354-e1bd-46ac-ad69-27f4f4236aa4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:40.409614 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876e3354-e1bd-46ac-ad69-27f4f4236aa4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:40.410998 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876e3354-e1bd-46ac-ad69-27f4f4236aa4 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:40.435003 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 496/1009] 10.222.0.50 () {60 vars in 1144 bytes} [Fri May 27 03:10:40 2022] POST /identity/v3/endpoints => generated 325 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:10:40.502046 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ad73e-c117-41f3-b946-2c45c05173c6 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:40.550131 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db7418-a2dd-49bc-ac09-3b7408e3dec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:40.550953 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db7418-a2dd-49bc-ac09-3b7408e3dec3 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:40.551236 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db7418-a2dd-49bc-ac09-3b7408e3dec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:40.551510 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db7418-a2dd-49bc-ac09-3b7408e3dec3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:40.552162 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77db7418-a2dd-49bc-ac09-3b7408e3dec3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:40.553618 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77db7418-a2dd-49bc-ac09-3b7408e3dec3 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:40.566942 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-77db7418-a2dd-49bc-ac09-3b7408e3dec3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 27 03:10:40.566942 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:40.566942 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:10:40.566942 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:10:40.566942 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:10:40.566942 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:10:40.566942 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:10:40.566942 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:40.566942 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 03:10:40.566942 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:40.566942 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:10:40.566942 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:40.566942 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:40.566942 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:40.566942 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:40.566942 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:40.566942 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:40.574191 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:40.574191 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:40.574191 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:40.574191 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:40.574191 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:40.574191 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:40.574191 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:10:40.574191 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:10:40.574191 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:10:40.574191 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:10:40.574191 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:40.574191 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:40.574191 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:40.574191 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:40.574191 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:40.574191 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:40.575228 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:40.575228 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:40.575228 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:40.575228 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:40.575228 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:40.575228 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:40.575228 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:40.575228 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:40.575228 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:40.575228 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:40.575228 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:10:40.575228 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:40.575228 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 03:10:40.575228 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:40.575228 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 514/1010] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:10:40 2022] GET /identity/v3/services/compute => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:10:40.577513 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876e3354-e1bd-46ac-ad69-27f4f4236aa4 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:40.693506 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876e3354-e1bd-46ac-ad69-27f4f4236aa4 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:40.723527 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:40.729736 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a529a2a-dcee-4dbe-93f5-5b025a9411bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:40.730773 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a529a2a-dcee-4dbe-93f5-5b025a9411bf None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:40.731301 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a529a2a-dcee-4dbe-93f5-5b025a9411bf None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:40.732056 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a529a2a-dcee-4dbe-93f5-5b025a9411bf None admin] PATH_INFO: `/v3/services` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:40.732770 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a529a2a-dcee-4dbe-93f5-5b025a9411bf None admin] RBAC: Adding query filter params (name=compute) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:40.733127 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a529a2a-dcee-4dbe-93f5-5b025a9411bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:40.734903 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a529a2a-dcee-4dbe-93f5-5b025a9411bf None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:40.745819 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 497/1011] 10.222.0.50 () {56 vars in 1112 bytes} [Fri May 27 03:10:40 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:40.747078 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db7418-a2dd-49bc-ac09-3b7408e3dec3 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:40.792765 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aff9ef-695f-48af-ae4d-b9ec5164031b None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:40.792991 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aff9ef-695f-48af-ae4d-b9ec5164031b None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:40.793560 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aff9ef-695f-48af-ae4d-b9ec5164031b None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:40.794962 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 498/1012] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:40.800575 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e2f1a6-e4ef-4d18-8c0b-6ca0f2b90bb4 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:40.800889 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e2f1a6-e4ef-4d18-8c0b-6ca0f2b90bb4 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:40.801200 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e2f1a6-e4ef-4d18-8c0b-6ca0f2b90bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:40.819866 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-07e2f1a6-e4ef-4d18-8c0b-6ca0f2b90bb4 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:40.821239 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-07e2f1a6-e4ef-4d18-8c0b-6ca0f2b90bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:40.821948 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:40.855836 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db7418-a2dd-49bc-ac09-3b7408e3dec3 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:40.891863 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 499/1013] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:40 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:10:40.893554 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff83f1d-3724-4ba2-a847-04b1e85df3b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:40.894330 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff83f1d-3724-4ba2-a847-04b1e85df3b7 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:40.894682 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff83f1d-3724-4ba2-a847-04b1e85df3b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:40.895023 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff83f1d-3724-4ba2-a847-04b1e85df3b7 None admin] PATH_INFO: `/v3/services` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:40.895798 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff83f1d-3724-4ba2-a847-04b1e85df3b7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:40.896170 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff83f1d-3724-4ba2-a847-04b1e85df3b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:40.897674 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff83f1d-3724-4ba2-a847-04b1e85df3b7 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:40.908854 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 515/1014] 10.222.0.50 () {56 vars in 1112 bytes} [Fri May 27 03:10:40 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:40.911325 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e2f1a6-e4ef-4d18-8c0b-6ca0f2b90bb4 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:40.917023 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff83f1d-3724-4ba2-a847-04b1e85df3b7 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:41.018211 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e2f1a6-e4ef-4d18-8c0b-6ca0f2b90bb4 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:41.023587 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff83f1d-3724-4ba2-a847-04b1e85df3b7 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:41.046088 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:41.056068 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59632c19-c3f0-4630-abad-68e6487279d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:41.056737 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59632c19-c3f0-4630-abad-68e6487279d3 None admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:41.057049 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59632c19-c3f0-4630-abad-68e6487279d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:41.057327 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59632c19-c3f0-4630-abad-68e6487279d3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:41.058071 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59632c19-c3f0-4630-abad-68e6487279d3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:41.059087 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbec2a1-f4ed-4b10-98f3-2494d28628d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:41.059449 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59632c19-c3f0-4630-abad-68e6487279d3 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:41.059751 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbec2a1-f4ed-4b10-98f3-2494d28628d2 None admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:41.060027 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbec2a1-f4ed-4b10-98f3-2494d28628d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:41.060350 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbec2a1-f4ed-4b10-98f3-2494d28628d2 None admin] PATH_INFO: `/v3/services` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:41.061047 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbec2a1-f4ed-4b10-98f3-2494d28628d2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:41.062397 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbec2a1-f4ed-4b10-98f3-2494d28628d2 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:41.064736 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:41.078228 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 516/1015] 10.222.0.50 () {60 vars in 1141 bytes} [Fri May 27 03:10:40 2022] POST /identity/v3/services => generated 242 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:10:41.082575 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 500/1016] 10.222.0.50 () {60 vars in 1144 bytes} [Fri May 27 03:10:40 2022] POST /identity/v3/endpoints => generated 334 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:10:42.690198 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370c322-2209-4850-9ff2-40335aa232c5 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:42.690777 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370c322-2209-4850-9ff2-40335aa232c5 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:42.690777 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370c322-2209-4850-9ff2-40335aa232c5 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:42.692372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 517/1017] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:42.697645 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bbb3cd-440a-446e-8c05-1b0928d331ce None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:42.698067 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bbb3cd-440a-446e-8c05-1b0928d331ce None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:42.698167 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bbb3cd-440a-446e-8c05-1b0928d331ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:42.715861 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-d1bbb3cd-440a-446e-8c05-1b0928d331ce None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:42.717082 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-d1bbb3cd-440a-446e-8c05-1b0928d331ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:42.795753 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 501/1018] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:42 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:10:42.819475 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370c322-2209-4850-9ff2-40335aa232c5 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:42.931365 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370c322-2209-4850-9ff2-40335aa232c5 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:42.966336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7db1b2c-4c64-4a8f-b497-0e8537d7a650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:42.967139 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db1b2c-4c64-4a8f-b497-0e8537d7a650 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:42.967434 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db1b2c-4c64-4a8f-b497-0e8537d7a650 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:42.967713 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7db1b2c-4c64-4a8f-b497-0e8537d7a650 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:42.968894 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db1b2c-4c64-4a8f-b497-0e8537d7a650 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:42.970376 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7db1b2c-4c64-4a8f-b497-0e8537d7a650 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:42.972266 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 518/1019] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:42.978196 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bbb3cd-440a-446e-8c05-1b0928d331ce None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:43.083202 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bbb3cd-440a-446e-8c05-1b0928d331ce None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:43.093202 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:43.122357 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e49046c-abb0-4861-8631-7610bc422177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:43.122357 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e49046c-abb0-4861-8631-7610bc422177 None admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:43.122357 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e49046c-abb0-4861-8631-7610bc422177 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:43.122738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e49046c-abb0-4861-8631-7610bc422177 None admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:43.123291 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e49046c-abb0-4861-8631-7610bc422177 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:43.124730 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e49046c-abb0-4861-8631-7610bc422177 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:43.156985 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 502/1020] 10.222.0.50 () {60 vars in 1141 bytes} [Fri May 27 03:10:42 2022] POST /identity/v3/projects => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:10:43.383605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca0543-7aae-4f20-b4f8-fcefc988f278 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:43.383935 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca0543-7aae-4f20-b4f8-fcefc988f278 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:43.384148 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca0543-7aae-4f20-b4f8-fcefc988f278 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:43.385752 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 519/1021] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:43.390917 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d08f03-4f6e-475e-857c-1a63bb92ea4e None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:43.391214 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d08f03-4f6e-475e-857c-1a63bb92ea4e None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:43.391486 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d08f03-4f6e-475e-857c-1a63bb92ea4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:43.395561 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43ce59-9dec-4089-9eb6-76dab4df0117 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:43.395844 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43ce59-9dec-4089-9eb6-76dab4df0117 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:43.396118 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43ce59-9dec-4089-9eb6-76dab4df0117 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:43.397657 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 520/1022] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:10:43.402524 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381b226-2432-423a-bc3f-c4526ec75690 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:43.405041 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381b226-2432-423a-bc3f-c4526ec75690 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:43.405338 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381b226-2432-423a-bc3f-c4526ec75690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:43.409967 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-14d08f03-4f6e-475e-857c-1a63bb92ea4e None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:43.411148 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-14d08f03-4f6e-475e-857c-1a63bb92ea4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:43.425014 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-4381b226-2432-423a-bc3f-c4526ec75690 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:43.432140 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-4381b226-2432-423a-bc3f-c4526ec75690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:43.484986 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 503/1023] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:10:43.501151 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:43.510054 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d08f03-4f6e-475e-857c-1a63bb92ea4e None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:43.514835 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 521/1024] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:43 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:10:43.537958 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4381b226-2432-423a-bc3f-c4526ec75690 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:43.611275 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:43.624848 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d08f03-4f6e-475e-857c-1a63bb92ea4e None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:43.639481 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4381b226-2432-423a-bc3f-c4526ec75690 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:43.662783 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecce482d-b4af-4d75-8584-40694c00c8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:43.664028 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecce482d-b4af-4d75-8584-40694c00c8f2 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:43.664539 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecce482d-b4af-4d75-8584-40694c00c8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:43.665016 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecce482d-b4af-4d75-8584-40694c00c8f2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:43.675536 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062376e-0200-4ca3-831c-4a6f41dd6228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:43.676268 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecce482d-b4af-4d75-8584-40694c00c8f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:43.676401 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062376e-0200-4ca3-831c-4a6f41dd6228 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:43.676594 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062376e-0200-4ca3-831c-4a6f41dd6228 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:43.676870 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062376e-0200-4ca3-831c-4a6f41dd6228 None admin] PATH_INFO: `/v3/services/image` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:43.677556 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d062376e-0200-4ca3-831c-4a6f41dd6228 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:43.678947 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d062376e-0200-4ca3-831c-4a6f41dd6228 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:43.685776 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-ecce482d-b4af-4d75-8584-40694c00c8f2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:10:43.685776 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:43.685776 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:43.685776 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:43.685776 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:43.685776 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:43.685776 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:43.685776 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:43.685776 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:43.685776 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:43.685776 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:43.685776 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:43.685776 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:43.685776 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:43.685776 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:43.685776 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:43.686703 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:43.686703 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:43.686703 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:43.686703 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:43.686703 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:43.686703 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:43.686703 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:43.686703 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:43.686703 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:43.686703 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:43.686703 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:43.686703 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:43.686703 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:43.686703 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:43.686703 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:43.686703 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:43.687538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:43.687538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:43.687538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:43.687538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:43.687538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:43.687538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:43.687538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:43.687538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:43.687538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:43.687538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:43.687538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:10:43.687538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:43.689809 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 504/1025] 10.222.0.50 () {56 vars in 1120 bytes} [Fri May 27 03:10:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:10:43.690740 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-d062376e-0200-4ca3-831c-4a6f41dd6228 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 27 03:10:43.690740 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:43.690740 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:10:43.690740 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:10:43.690740 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:10:43.690740 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:10:43.690740 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:10:43.690740 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:43.690740 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 03:10:43.690740 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:43.690740 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:10:43.690740 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:43.690740 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:43.690740 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:43.690740 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:43.690740 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:43.690740 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:43.691977 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:43.691977 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:43.691977 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:43.691977 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:43.691977 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:43.691977 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:43.691977 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:10:43.691977 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:10:43.691977 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:10:43.691977 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:10:43.691977 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:43.691977 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:43.691977 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:43.691977 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:43.691977 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:43.691977 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:43.698888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:43.698888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:43.698888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:43.698888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:43.698888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:43.698888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:43.698888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:43.698888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:43.698888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:43.698888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:43.698888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:10:43.698888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:43.698888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 03:10:43.698888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:43.698888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 522/1026] 10.222.0.50 () {56 vars in 1105 bytes} [Fri May 27 03:10:43 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:43.700687 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecce482d-b4af-4d75-8584-40694c00c8f2 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:43.701197 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062376e-0200-4ca3-831c-4a6f41dd6228 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:43.759439 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:43.806003 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062376e-0200-4ca3-831c-4a6f41dd6228 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:43.806253 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecce482d-b4af-4d75-8584-40694c00c8f2 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:43.842083 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89490e8-42bb-4083-923c-262a3ee00947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:43.842436 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c027c00-a3ad-4fd6-bbb4-ae8e0ebfd000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:43.843126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c027c00-a3ad-4fd6-bbb4-ae8e0ebfd000 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:43.843405 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c027c00-a3ad-4fd6-bbb4-ae8e0ebfd000 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:43.843679 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c027c00-a3ad-4fd6-bbb4-ae8e0ebfd000 None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:43.843930 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89490e8-42bb-4083-923c-262a3ee00947 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:43.844386 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89490e8-42bb-4083-923c-262a3ee00947 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:43.844486 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c027c00-a3ad-4fd6-bbb4-ae8e0ebfd000 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:43.844724 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c027c00-a3ad-4fd6-bbb4-ae8e0ebfd000 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:43.845178 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89490e8-42bb-4083-923c-262a3ee00947 None admin] PATH_INFO: `/v3/services` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:43.846110 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89490e8-42bb-4083-923c-262a3ee00947 None admin] RBAC: Adding query filter params (name=image) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:43.846373 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c027c00-a3ad-4fd6-bbb4-ae8e0ebfd000 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:43.846813 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89490e8-42bb-4083-923c-262a3ee00947 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:43.848372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89490e8-42bb-4083-923c-262a3ee00947 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:43.857005 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 505/1027] 10.222.0.50 () {56 vars in 1115 bytes} [Fri May 27 03:10:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:43.858600 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 523/1028] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:43.859995 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c027c00-a3ad-4fd6-bbb4-ae8e0ebfd000 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:43.860402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89490e8-42bb-4083-923c-262a3ee00947 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:43.878034 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:43.967668 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89490e8-42bb-4083-923c-262a3ee00947 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:43.972688 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c027c00-a3ad-4fd6-bbb4-ae8e0ebfd000 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:43.992505 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:44.009166 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926eddf-7227-4e3c-9660-25e1c369b990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:44.009717 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d3b55-8f39-419d-8f5b-f92dd8e97ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:44.009956 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926eddf-7227-4e3c-9660-25e1c369b990 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:44.010244 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926eddf-7227-4e3c-9660-25e1c369b990 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:44.010368 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d3b55-8f39-419d-8f5b-f92dd8e97ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:44.010535 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926eddf-7227-4e3c-9660-25e1c369b990 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:44.010629 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d3b55-8f39-419d-8f5b-f92dd8e97ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:44.010911 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25d3b55-8f39-419d-8f5b-f92dd8e97ba7 None admin] PATH_INFO: `/v3/services` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:44.011628 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25d3b55-8f39-419d-8f5b-f92dd8e97ba7 None admin] RBAC: Adding query filter params (type=image) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:44.011930 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25d3b55-8f39-419d-8f5b-f92dd8e97ba7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:44.013312 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25d3b55-8f39-419d-8f5b-f92dd8e97ba7 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:44.023238 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 506/1029] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:43 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:44.024373 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d3b55-8f39-419d-8f5b-f92dd8e97ba7 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:44.027944 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5926eddf-7227-4e3c-9660-25e1c369b990 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:44.029500 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5926eddf-7227-4e3c-9660-25e1c369b990 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:44.048147 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-5926eddf-7227-4e3c-9660-25e1c369b990 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:44.048147 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:44.048147 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:44.048147 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:44.048147 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:44.048147 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:44.048147 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:44.048147 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:44.048147 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:44.048147 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:44.048147 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:44.048147 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:44.048147 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:44.048147 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:44.048147 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:44.049166 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:44.049166 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:44.049166 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:44.049166 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:44.049166 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:44.049166 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:44.049166 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:44.049166 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:44.049166 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:44.049166 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:44.049166 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:44.049166 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:44.049166 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:44.049166 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:44.049166 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:44.049166 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:44.049166 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:44.050043 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:44.050043 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:44.050043 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:44.050043 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:44.050043 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 524/1030] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:44.055576 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926eddf-7227-4e3c-9660-25e1c369b990 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:44.117978 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:44.148357 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25d3b55-8f39-419d-8f5b-f92dd8e97ba7 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:44.159876 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926eddf-7227-4e3c-9660-25e1c369b990 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:44.184417 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57827fef-9593-4a82-b03f-dc8be18ada94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:44.185119 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57827fef-9593-4a82-b03f-dc8be18ada94 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:44.185406 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57827fef-9593-4a82-b03f-dc8be18ada94 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:44.185685 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57827fef-9593-4a82-b03f-dc8be18ada94 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:44.186485 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57827fef-9593-4a82-b03f-dc8be18ada94 None admin] RBAC: Adding query filter params (interface=public, service_id=dabce5f819d44c1694264fe5d9236900, region_id=RegionOne) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:44.186797 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57827fef-9593-4a82-b03f-dc8be18ada94 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:44.188183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57827fef-9593-4a82-b03f-dc8be18ada94 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:44.193425 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 507/1031] 10.222.0.50 () {56 vars in 1251 bytes} [Fri May 27 03:10:44 2022] GET /identity/v3/endpoints?service_id=dabce5f819d44c1694264fe5d9236900&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:44.199204 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57827fef-9593-4a82-b03f-dc8be18ada94 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:44.201692 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae169e3-655f-4df3-afb2-455e6bc630ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:44.202448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae169e3-655f-4df3-afb2-455e6bc630ad None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:44.202731 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae169e3-655f-4df3-afb2-455e6bc630ad None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:44.203007 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae169e3-655f-4df3-afb2-455e6bc630ad None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:44.203730 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae169e3-655f-4df3-afb2-455e6bc630ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:44.204038 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae169e3-655f-4df3-afb2-455e6bc630ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:44.205500 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae169e3-655f-4df3-afb2-455e6bc630ad None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:44.218607 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 525/1032] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:10:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:44.224927 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae169e3-655f-4df3-afb2-455e6bc630ad None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:44.257659 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:44.307615 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57827fef-9593-4a82-b03f-dc8be18ada94 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:44.344114 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae169e3-655f-4df3-afb2-455e6bc630ad None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:44.345317 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6653b81e-057e-4463-ac55-eb99b85e131b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:44.346082 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653b81e-057e-4463-ac55-eb99b85e131b None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:44.346364 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653b81e-057e-4463-ac55-eb99b85e131b None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:44.346640 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653b81e-057e-4463-ac55-eb99b85e131b None admin] PATH_INFO: `/v3/services` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:44.347331 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6653b81e-057e-4463-ac55-eb99b85e131b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:44.348718 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6653b81e-057e-4463-ac55-eb99b85e131b None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:44.355715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 508/1033] 10.222.0.50 () {56 vars in 1087 bytes} [Fri May 27 03:10:44 2022] GET /identity/v3/services => generated 1957 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:10:44.383851 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b9d6a-6c80-4a6f-9eb4-13516814bc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:44.384681 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b9d6a-6c80-4a6f-9eb4-13516814bc04 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:44.384993 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b9d6a-6c80-4a6f-9eb4-13516814bc04 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:44.385265 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b9d6a-6c80-4a6f-9eb4-13516814bc04 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:44.394826 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817b9d6a-6c80-4a6f-9eb4-13516814bc04 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:44.396337 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817b9d6a-6c80-4a6f-9eb4-13516814bc04 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:44.413275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-817b9d6a-6c80-4a6f-9eb4-13516814bc04 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 03:10:44.413275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:44.413275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:44.413275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:44.413275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:44.413275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:44.413275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:44.413275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:44.413275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:44.413275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:44.413275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:44.413275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:44.413275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:10:44.413275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:10:44.413275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:10:44.413275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:10:44.414267 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:44.414267 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:44.414267 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:10:44.414267 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:44.414267 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:10:44.414267 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:44.414267 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:44.414267 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:44.414267 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:44.414267 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:44.414267 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:44.414267 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:44.414267 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:44.414267 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:44.414267 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:44.414267 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:44.414267 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:44.414267 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:44.415218 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:44.415218 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:44.415218 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:10:44.415218 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:10:44.415218 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:10:44.415218 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:10:44.415218 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:10:44.415218 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:10:44.415218 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 03:10:44.415218 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:44.417360 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 526/1034] 10.222.0.50 () {56 vars in 1093 bytes} [Fri May 27 03:10:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:10:44.420478 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6653b81e-057e-4463-ac55-eb99b85e131b None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:44.425311 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:44.539646 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6653b81e-057e-4463-ac55-eb99b85e131b None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:44.579630 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9a2ed2-b92b-4c5c-8af4-ed188f28c0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:44.581063 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9a2ed2-b92b-4c5c-8af4-ed188f28c0f8 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:44.581599 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9a2ed2-b92b-4c5c-8af4-ed188f28c0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:44.582075 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9a2ed2-b92b-4c5c-8af4-ed188f28c0f8 None admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:44.583117 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9a2ed2-b92b-4c5c-8af4-ed188f28c0f8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:44.583640 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9a2ed2-b92b-4c5c-8af4-ed188f28c0f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:44.585301 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9a2ed2-b92b-4c5c-8af4-ed188f28c0f8 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:44.604246 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 509/1035] 10.222.0.50 () {56 vars in 1133 bytes} [Fri May 27 03:10:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:44.610029 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b9d6a-6c80-4a6f-9eb4-13516814bc04 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:44.705907 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:44.716835 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b9d6a-6c80-4a6f-9eb4-13516814bc04 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:44.751435 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c979fb7b-65da-423f-9e7c-5f5f06d2b28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:44.752287 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979fb7b-65da-423f-9e7c-5f5f06d2b28c None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:44.752586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979fb7b-65da-423f-9e7c-5f5f06d2b28c None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:44.752852 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979fb7b-65da-423f-9e7c-5f5f06d2b28c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:44.821128 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c979fb7b-65da-423f-9e7c-5f5f06d2b28c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:44.822592 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c979fb7b-65da-423f-9e7c-5f5f06d2b28c None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:44.917028 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:44.932161 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 527/1036] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:10:44 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:10:44.933556 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9a2ed2-b92b-4c5c-8af4-ed188f28c0f8 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:44.999490 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:45.045222 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9a2ed2-b92b-4c5c-8af4-ed188f28c0f8 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:45.083559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff68a4de-9723-41eb-9081-3250d2fbc747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:45.089366 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68a4de-9723-41eb-9081-3250d2fbc747 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:45.089666 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68a4de-9723-41eb-9081-3250d2fbc747 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:45.089948 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68a4de-9723-41eb-9081-3250d2fbc747 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:45.102018 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff68a4de-9723-41eb-9081-3250d2fbc747 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:45.103477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff68a4de-9723-41eb-9081-3250d2fbc747 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:45.126284 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-ff68a4de-9723-41eb-9081-3250d2fbc747 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:45.126284 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:45.126284 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:45.126284 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:45.126284 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:45.126284 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:45.126284 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:45.126284 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:45.126284 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:45.126284 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:45.126284 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:45.126284 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:45.126284 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:45.126284 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:45.126284 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:45.127714 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:45.127714 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:45.127714 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:45.127714 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:45.127714 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:45.127714 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:45.127714 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:45.127714 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:45.127714 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:45.127714 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:45.127714 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:45.127714 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:45.127714 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:45.127714 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:45.127714 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:45.127714 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:45.127714 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:45.128627 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:45.128627 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:45.128627 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:45.128627 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:45.128627 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 510/1037] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:45.134709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c979fb7b-65da-423f-9e7c-5f5f06d2b28c None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:45.253466 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c979fb7b-65da-423f-9e7c-5f5f06d2b28c None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:45.297052 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e76360-c12f-4327-8663-761b738ae63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:45.297727 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e76360-c12f-4327-8663-761b738ae63e None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:45.298005 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e76360-c12f-4327-8663-761b738ae63e None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:45.298277 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e76360-c12f-4327-8663-761b738ae63e None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:45.299803 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e76360-c12f-4327-8663-761b738ae63e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:45.300002 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e76360-c12f-4327-8663-761b738ae63e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:45.301448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e76360-c12f-4327-8663-761b738ae63e None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:45.320942 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 528/1038] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:10:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:45.326629 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff68a4de-9723-41eb-9081-3250d2fbc747 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:45.424554 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d661c3ef-183b-4c0c-baeb-445849f3afa4 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:45.424554 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d661c3ef-183b-4c0c-baeb-445849f3afa4 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:45.424927 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d661c3ef-183b-4c0c-baeb-445849f3afa4 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:45.426472 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 529/1039] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:45.431344 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4182e-a770-4e96-a6be-7caa98965981 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:45.431628 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4182e-a770-4e96-a6be-7caa98965981 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:45.432099 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b4182e-a770-4e96-a6be-7caa98965981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:45.437037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff68a4de-9723-41eb-9081-3250d2fbc747 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:45.449582 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-60b4182e-a770-4e96-a6be-7caa98965981 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:45.450755 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-60b4182e-a770-4e96-a6be-7caa98965981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:45.452039 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:45.479196 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf91e76f-df4b-4aaa-b847-6141aad7722a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:45.479941 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf91e76f-df4b-4aaa-b847-6141aad7722a None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:45.480215 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf91e76f-df4b-4aaa-b847-6141aad7722a None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:45.480549 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf91e76f-df4b-4aaa-b847-6141aad7722a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:45.492571 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf91e76f-df4b-4aaa-b847-6141aad7722a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:45.494938 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf91e76f-df4b-4aaa-b847-6141aad7722a None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:45.507414 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:45.511294 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-cf91e76f-df4b-4aaa-b847-6141aad7722a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:45.511294 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:45.511294 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:45.511294 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:45.511294 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:45.511294 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:45.511294 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:45.511294 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:45.511294 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:45.511294 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:45.511294 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:45.511294 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:45.511294 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:10:45.511294 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:10:45.511294 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:10:45.512195 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:10:45.512195 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:45.512195 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:45.512195 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:45.512195 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:45.512195 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:45.512195 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:45.512195 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:45.512195 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:45.512195 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:45.512195 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:45.512195 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:45.512195 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:45.512195 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:45.512195 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:45.512195 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:45.512195 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:45.513138 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:10:45.513138 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:10:45.513138 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:10:45.513138 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:10:45.513138 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:10:45.513138 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:10:45.513138 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:45.513138 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:45.517950 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 511/1040] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:10:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:45.519409 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf91e76f-df4b-4aaa-b847-6141aad7722a None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:45.530169 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 530/1041] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:45 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:10:45.554777 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b4182e-a770-4e96-a6be-7caa98965981 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:45.630343 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf91e76f-df4b-4aaa-b847-6141aad7722a None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:45.662571 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b4182e-a770-4e96-a6be-7caa98965981 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:45.666386 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac494e4-d079-4378-84a7-389744210ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:45.667055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac494e4-d079-4378-84a7-389744210ab2 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:45.667332 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac494e4-d079-4378-84a7-389744210ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:45.667600 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac494e4-d079-4378-84a7-389744210ab2 None admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:45.668384 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac494e4-d079-4378-84a7-389744210ab2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:45.668691 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac494e4-d079-4378-84a7-389744210ab2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:45.670679 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac494e4-d079-4378-84a7-389744210ab2 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:45.688162 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 512/1042] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 03:10:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:45.693898 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac494e4-d079-4378-84a7-389744210ab2 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:45.700083 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c168a-5b78-492b-b7cd-e912dc59e917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:45.700892 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c168a-5b78-492b-b7cd-e912dc59e917 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:45.701203 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c168a-5b78-492b-b7cd-e912dc59e917 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:45.701477 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c168a-5b78-492b-b7cd-e912dc59e917 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:45.703161 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420c168a-5b78-492b-b7cd-e912dc59e917 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:45.705128 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420c168a-5b78-492b-b7cd-e912dc59e917 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:45.707395 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 531/1043] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:45.712810 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c168a-5b78-492b-b7cd-e912dc59e917 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:45.730340 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:45.766392 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:45.800392 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac494e4-d079-4378-84a7-389744210ab2 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:45.821762 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c168a-5b78-492b-b7cd-e912dc59e917 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:45.835220 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853141c3-4193-4692-8727-0262d1121ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:45.835875 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853141c3-4193-4692-8727-0262d1121ee7 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:45.836151 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853141c3-4193-4692-8727-0262d1121ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:45.836606 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853141c3-4193-4692-8727-0262d1121ee7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:45.837265 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853141c3-4193-4692-8727-0262d1121ee7 None admin] RBAC: Adding query filter params (role.id=8c199022919d4ef2b185859d8e1dc9a9, scope.project.id=f8398941252346468483ea0c863122f4, user.id=1a699988f08a4001b3b7d764f01f4fc5) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:45.837572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853141c3-4193-4692-8727-0262d1121ee7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:45.839165 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853141c3-4193-4692-8727-0262d1121ee7 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:45.844643 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 513/1044] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:10:45 2022] GET /identity/v3/role_assignments?user.id=1a699988f08a4001b3b7d764f01f4fc5&scope.project.id=f8398941252346468483ea0c863122f4&role.id=8c199022919d4ef2b185859d8e1dc9a9 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:45.858053 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cdf51c-30b7-4cde-b556-ddcb5dfef15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:45.858780 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cdf51c-30b7-4cde-b556-ddcb5dfef15f None admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:45.859064 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cdf51c-30b7-4cde-b556-ddcb5dfef15f None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:45.859349 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cdf51c-30b7-4cde-b556-ddcb5dfef15f None admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:45.860081 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cdf51c-30b7-4cde-b556-ddcb5dfef15f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:45.861682 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cdf51c-30b7-4cde-b556-ddcb5dfef15f None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:45.893001 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 532/1045] 10.222.0.50 () {60 vars in 1132 bytes} [Fri May 27 03:10:45 2022] POST /identity/v3/users => generated 285 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:10:46.569170 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b2a99-9d05-4f55-afe4-27eb74cadac4 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:46.569674 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b2a99-9d05-4f55-afe4-27eb74cadac4 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:46.569674 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b2a99-9d05-4f55-afe4-27eb74cadac4 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:46.571206 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 514/1046] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:46.576746 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01af794-e991-4c73-84fa-d2786318c0df None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:46.577073 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01af794-e991-4c73-84fa-d2786318c0df None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:46.577358 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01af794-e991-4c73-84fa-d2786318c0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:46.594914 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-d01af794-e991-4c73-84fa-d2786318c0df None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:46.596154 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-d01af794-e991-4c73-84fa-d2786318c0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:46.677304 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 533/1047] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:46 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:10:46.696031 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934b2a99-9d05-4f55-afe4-27eb74cadac4 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:46.808231 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:46.813842 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934b2a99-9d05-4f55-afe4-27eb74cadac4 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:46.848526 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f01e20-b680-4d21-8f36-94280ec2e5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:46.849279 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f01e20-b680-4d21-8f36-94280ec2e5b6 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:46.849554 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f01e20-b680-4d21-8f36-94280ec2e5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:46.849815 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f01e20-b680-4d21-8f36-94280ec2e5b6 None admin] PATH_INFO: `/v3/services/image` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:46.850454 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f01e20-b680-4d21-8f36-94280ec2e5b6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:46.851810 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f01e20-b680-4d21-8f36-94280ec2e5b6 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:46.863348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-a5f01e20-b680-4d21-8f36-94280ec2e5b6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 27 03:10:46.863348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:46.863348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:10:46.863348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:10:46.863348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:10:46.863348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:10:46.863348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:10:46.863348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:46.863348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 03:10:46.863348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:46.863348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:10:46.863348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:46.863348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:46.863348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:46.863348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:46.863348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:46.863348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:46.864320 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:46.864320 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:46.864320 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:46.864320 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:46.864320 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:46.864320 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:46.864320 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:10:46.864320 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:10:46.864320 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:10:46.864320 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:10:46.864320 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:46.864320 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:46.864320 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:46.864320 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:46.864320 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:46.864320 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:46.870494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:46.870494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:46.870494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:46.870494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:46.870494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:46.870494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:46.870494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:46.870494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:46.870494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:46.870494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:46.870494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:10:46.870494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:10:46.870494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 03:10:46.870494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:46.870494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 515/1048] 10.222.0.50 () {56 vars in 1105 bytes} [Fri May 27 03:10:46 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:46.871344 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01af794-e991-4c73-84fa-d2786318c0df None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:46.906476 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:46.974949 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01af794-e991-4c73-84fa-d2786318c0df None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:47.009803 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77158414-250a-4523-8043-0394e46a6dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:47.010580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77158414-250a-4523-8043-0394e46a6dcd None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:47.010854 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77158414-250a-4523-8043-0394e46a6dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:47.011124 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77158414-250a-4523-8043-0394e46a6dcd None admin] PATH_INFO: `/v3/services` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:47.011822 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77158414-250a-4523-8043-0394e46a6dcd None admin] RBAC: Adding query filter params (name=image) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:47.012122 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77158414-250a-4523-8043-0394e46a6dcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:47.013534 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77158414-250a-4523-8043-0394e46a6dcd None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:47.018793 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 534/1049] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:47.024307 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f01e20-b680-4d21-8f36-94280ec2e5b6 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:47.132373 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f01e20-b680-4d21-8f36-94280ec2e5b6 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:47.169885 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56483441-3491-49dc-83bb-649dce886b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:47.170553 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56483441-3491-49dc-83bb-649dce886b5e None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:47.170832 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56483441-3491-49dc-83bb-649dce886b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:47.171102 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56483441-3491-49dc-83bb-649dce886b5e None admin] PATH_INFO: `/v3/services` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:47.171804 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56483441-3491-49dc-83bb-649dce886b5e None admin] RBAC: Adding query filter params (type=image) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:47.172104 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56483441-3491-49dc-83bb-649dce886b5e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:47.173535 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56483441-3491-49dc-83bb-649dce886b5e None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:47.183567 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 516/1050] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:47 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:47.185030 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77158414-250a-4523-8043-0394e46a6dcd None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:47.288851 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:47.292686 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77158414-250a-4523-8043-0394e46a6dcd None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:47.330189 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344333e-778e-40fe-a3fa-bd5b8b297820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:47.330849 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344333e-778e-40fe-a3fa-bd5b8b297820 None admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:47.331123 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344333e-778e-40fe-a3fa-bd5b8b297820 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:47.331390 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344333e-778e-40fe-a3fa-bd5b8b297820 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:47.332133 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7344333e-778e-40fe-a3fa-bd5b8b297820 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:47.333549 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7344333e-778e-40fe-a3fa-bd5b8b297820 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:47.360977 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 535/1051] 10.222.0.50 () {60 vars in 1144 bytes} [Fri May 27 03:10:47 2022] POST /identity/v3/endpoints => generated 327 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:10:48.054748 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00b60d-ac45-46ae-a813-b075dbe1e96b None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:48.055300 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00b60d-ac45-46ae-a813-b075dbe1e96b None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:48.055300 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00b60d-ac45-46ae-a813-b075dbe1e96b None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:48.056821 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 517/1052] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:48.062043 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae2dba-53fd-4650-bb4d-4192dbabae93 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:48.062490 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae2dba-53fd-4650-bb4d-4192dbabae93 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:48.062569 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae2dba-53fd-4650-bb4d-4192dbabae93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:48.080235 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-57ae2dba-53fd-4650-bb4d-4192dbabae93 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:48.081529 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-57ae2dba-53fd-4650-bb4d-4192dbabae93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:48.160990 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 536/1053] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:10:48.169486 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f18ed-eda0-41d5-8f29-480cc778419a None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:48.169772 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f18ed-eda0-41d5-8f29-480cc778419a None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:48.170043 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f18ed-eda0-41d5-8f29-480cc778419a None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:48.171518 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 518/1054] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:48.176710 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada09f50-cd93-4401-8fe4-00bd11c82b6b None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:48.177040 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada09f50-cd93-4401-8fe4-00bd11c82b6b None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:48.177317 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada09f50-cd93-4401-8fe4-00bd11c82b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:48.183331 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16f18ed-eda0-41d5-8f29-480cc778419a None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:48.194840 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-ada09f50-cd93-4401-8fe4-00bd11c82b6b None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:48.196026 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-ada09f50-cd93-4401-8fe4-00bd11c82b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:48.213454 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:48.266750 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 537/1055] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:10:48.290240 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada09f50-cd93-4401-8fe4-00bd11c82b6b None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:48.303678 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16f18ed-eda0-41d5-8f29-480cc778419a None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:48.305813 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:48.339954 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b625da6b-2541-4c57-a584-51bf404dfa30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:48.340777 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625da6b-2541-4c57-a584-51bf404dfa30 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:48.341079 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625da6b-2541-4c57-a584-51bf404dfa30 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:48.341351 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625da6b-2541-4c57-a584-51bf404dfa30 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:48.351948 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b625da6b-2541-4c57-a584-51bf404dfa30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:48.360087 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-b625da6b-2541-4c57-a584-51bf404dfa30 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:10:48.360087 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:48.360087 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:48.360087 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:48.360087 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:48.360087 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:48.360087 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:48.360087 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:48.360087 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:48.360087 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:48.360087 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:48.360087 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:48.360087 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:48.360087 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:48.360087 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:48.360087 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:48.361166 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:48.361166 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:48.361166 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:48.361166 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:48.361166 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:48.361166 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:48.361166 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:48.361166 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:48.361166 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:48.361166 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:48.361166 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:48.361166 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:48.361166 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:48.361166 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:48.361166 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:48.361166 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:48.362029 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:48.362029 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:48.362029 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:48.362029 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:48.362029 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:48.362029 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:48.362029 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:48.362029 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:48.362029 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:48.362029 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:48.362029 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:10:48.362029 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:48.362029 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 519/1056] 10.222.0.50 () {56 vars in 1120 bytes} [Fri May 27 03:10:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:48.367254 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b625da6b-2541-4c57-a584-51bf404dfa30 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:48.393554 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada09f50-cd93-4401-8fe4-00bd11c82b6b None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:48.433400 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a003c7-e803-4529-aba3-f9daa0a41452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:48.434501 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a003c7-e803-4529-aba3-f9daa0a41452 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:48.435702 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a003c7-e803-4529-aba3-f9daa0a41452 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:48.435988 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a003c7-e803-4529-aba3-f9daa0a41452 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:48.450550 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a003c7-e803-4529-aba3-f9daa0a41452 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:48.458101 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:48.459027 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-d2a003c7-e803-4529-aba3-f9daa0a41452 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:10:48.459027 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:48.459027 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:48.459027 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:48.459027 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:48.459027 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:48.459027 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:48.459027 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:48.459027 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:48.459027 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:48.459027 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:48.459027 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:48.459027 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:48.459027 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:48.459027 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:48.459027 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:48.459941 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:48.459941 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:48.459941 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:48.459941 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:48.459941 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:48.459941 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:48.459941 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:48.459941 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:48.459941 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:48.459941 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:48.459941 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:48.459941 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:48.459941 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:48.459941 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:48.459941 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:48.459941 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:48.460787 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:48.460787 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:48.460787 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:48.460787 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:48.460787 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:48.460787 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:48.460787 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:48.460787 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:48.460787 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:48.460787 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:48.460787 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:10:48.460787 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:48.460787 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 538/1057] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 03:10:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:10:48.466146 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a003c7-e803-4529-aba3-f9daa0a41452 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:48.478849 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b625da6b-2541-4c57-a584-51bf404dfa30 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:48.527818 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b732f0-3b7a-4345-8d96-8ba3a15c748a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:48.528501 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b732f0-3b7a-4345-8d96-8ba3a15c748a None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:48.528778 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b732f0-3b7a-4345-8d96-8ba3a15c748a None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:48.529076 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b732f0-3b7a-4345-8d96-8ba3a15c748a None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:48.529826 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b732f0-3b7a-4345-8d96-8ba3a15c748a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:48.530129 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b732f0-3b7a-4345-8d96-8ba3a15c748a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:48.531505 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b732f0-3b7a-4345-8d96-8ba3a15c748a None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:48.540364 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 520/1058] 10.222.0.50 () {56 vars in 1115 bytes} [Fri May 27 03:10:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:48.546446 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b732f0-3b7a-4345-8d96-8ba3a15c748a None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:48.555625 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:48.591377 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a003c7-e803-4529-aba3-f9daa0a41452 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:48.627162 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1825fbc1-66ad-40bf-bc4d-7cf1ae0c08e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:48.627807 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825fbc1-66ad-40bf-bc4d-7cf1ae0c08e9 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:48.628084 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825fbc1-66ad-40bf-bc4d-7cf1ae0c08e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:48.628353 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825fbc1-66ad-40bf-bc4d-7cf1ae0c08e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:48.629131 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1825fbc1-66ad-40bf-bc4d-7cf1ae0c08e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:48.629624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1825fbc1-66ad-40bf-bc4d-7cf1ae0c08e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:48.630815 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1825fbc1-66ad-40bf-bc4d-7cf1ae0c08e9 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:48.644510 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 539/1059] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:10:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:48.645925 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1825fbc1-66ad-40bf-bc4d-7cf1ae0c08e9 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:48.655679 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b732f0-3b7a-4345-8d96-8ba3a15c748a None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:48.692006 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a7f82-0c85-43fb-9b3a-794e2fe8683c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:48.692812 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a7f82-0c85-43fb-9b3a-794e2fe8683c None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:48.693133 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a7f82-0c85-43fb-9b3a-794e2fe8683c None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:48.693408 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a7f82-0c85-43fb-9b3a-794e2fe8683c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:48.704258 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623a7f82-0c85-43fb-9b3a-794e2fe8683c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:48.706049 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623a7f82-0c85-43fb-9b3a-794e2fe8683c None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:48.714947 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:48.726231 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-623a7f82-0c85-43fb-9b3a-794e2fe8683c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:48.726231 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:48.726231 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:48.726231 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:48.726231 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:48.726231 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:48.726231 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:48.726231 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:48.726231 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:48.726231 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:48.726231 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:48.726231 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:48.726231 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:48.726231 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:48.726231 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:48.727355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:48.727355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:48.727355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:48.727355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:48.727355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:48.727355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:48.727355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:48.727355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:48.727355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:48.727355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:48.727355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:48.727355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:48.727355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:48.727355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:48.727355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:48.727355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:48.727355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:48.728648 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:48.728648 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:48.728648 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:48.728648 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:48.728648 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 521/1060] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:48.733163 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a7f82-0c85-43fb-9b3a-794e2fe8683c None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:48.763726 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1825fbc1-66ad-40bf-bc4d-7cf1ae0c08e9 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:48.799424 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2fd6fe-6a0f-4726-b68c-16497551d2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:48.800307 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2fd6fe-6a0f-4726-b68c-16497551d2cf None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:48.800586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2fd6fe-6a0f-4726-b68c-16497551d2cf None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:48.800858 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2fd6fe-6a0f-4726-b68c-16497551d2cf None admin] PATH_INFO: `/v3/users/6fe9e3df88d44547a5999968c0b3f2e5` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:48.802840 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:48.815395 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2fd6fe-6a0f-4726-b68c-16497551d2cf None admin] RBAC: Authorizing `identity:get_user(user_id=6fe9e3df88d44547a5999968c0b3f2e5)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:48.816854 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2fd6fe-6a0f-4726-b68c-16497551d2cf None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:48.818602 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 540/1061] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:10:48 2022] GET /identity/v3/users/6fe9e3df88d44547a5999968c0b3f2e5 => generated 285 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:48.824185 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2fd6fe-6a0f-4726-b68c-16497551d2cf None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:48.845092 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a7f82-0c85-43fb-9b3a-794e2fe8683c None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:48.881613 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d59b7b-05e7-48c3-a7c2-d3ffbf8dc33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:48.882285 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d59b7b-05e7-48c3-a7c2-d3ffbf8dc33f None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:48.882576 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d59b7b-05e7-48c3-a7c2-d3ffbf8dc33f None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:48.882844 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d59b7b-05e7-48c3-a7c2-d3ffbf8dc33f None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:48.883558 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d59b7b-05e7-48c3-a7c2-d3ffbf8dc33f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:48.883976 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d59b7b-05e7-48c3-a7c2-d3ffbf8dc33f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:48.885566 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d59b7b-05e7-48c3-a7c2-d3ffbf8dc33f None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:48.898562 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 522/1062] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:10:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:48.904083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d59b7b-05e7-48c3-a7c2-d3ffbf8dc33f None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:48.931368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2fd6fe-6a0f-4726-b68c-16497551d2cf None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:48.968922 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:48.971817 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7028b6c-3eea-4bd6-9d13-8d5fa9f6033a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:48.972485 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7028b6c-3eea-4bd6-9d13-8d5fa9f6033a None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:48.972767 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7028b6c-3eea-4bd6-9d13-8d5fa9f6033a None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:48.973068 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7028b6c-3eea-4bd6-9d13-8d5fa9f6033a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:49.016737 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d59b7b-05e7-48c3-a7c2-d3ffbf8dc33f None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:49.039110 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:49.051163 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7028b6c-3eea-4bd6-9d13-8d5fa9f6033a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:49.053066 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7028b6c-3eea-4bd6-9d13-8d5fa9f6033a None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:49.055301 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f9df8-964f-488d-aaab-4ca83d03f122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:49.056096 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f9df8-964f-488d-aaab-4ca83d03f122 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:49.056373 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f9df8-964f-488d-aaab-4ca83d03f122 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:49.056642 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f9df8-964f-488d-aaab-4ca83d03f122 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:49.064247 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f9df8-964f-488d-aaab-4ca83d03f122 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:49.065713 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f9df8-964f-488d-aaab-4ca83d03f122 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:49.081037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-773f9df8-964f-488d-aaab-4ca83d03f122 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 03:10:49.081037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:49.081037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:49.081037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:49.081037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:49.081037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:49.081037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:49.081037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:49.081037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:49.081037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:49.081037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:49.081037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:49.081037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:10:49.081037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:10:49.081037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:10:49.081037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:10:49.082037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:49.082037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:49.082037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:10:49.082037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:49.082037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:10:49.082037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:49.082037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:49.082037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:49.082037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:49.082037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:49.082037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:49.082037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:49.082037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:49.082037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:49.082037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:49.082037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:49.082037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:49.082037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:49.083826 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:49.083826 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:49.083826 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:10:49.083826 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:10:49.083826 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:10:49.083826 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:10:49.083826 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:10:49.083826 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:10:49.083826 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 03:10:49.083826 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:49.083826 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 523/1063] 10.222.0.50 () {56 vars in 1093 bytes} [Fri May 27 03:10:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:10:49.087937 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f9df8-964f-488d-aaab-4ca83d03f122 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:49.168241 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 541/1064] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:10:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 03:10:49.171289 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7028b6c-3eea-4bd6-9d13-8d5fa9f6033a None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:49.201070 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f9df8-964f-488d-aaab-4ca83d03f122 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:49.223841 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:49.240724 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a72055-cc21-49fe-a6f6-242acadebc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:49.241434 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a72055-cc21-49fe-a6f6-242acadebc69 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:49.241715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a72055-cc21-49fe-a6f6-242acadebc69 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:49.242011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a72055-cc21-49fe-a6f6-242acadebc69 None admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:49.243338 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a72055-cc21-49fe-a6f6-242acadebc69 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:49.243338 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a72055-cc21-49fe-a6f6-242acadebc69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:49.244570 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a72055-cc21-49fe-a6f6-242acadebc69 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:49.267182 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 524/1065] 10.222.0.50 () {56 vars in 1133 bytes} [Fri May 27 03:10:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:49.267487 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:49.268410 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a72055-cc21-49fe-a6f6-242acadebc69 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:49.279004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7028b6c-3eea-4bd6-9d13-8d5fa9f6033a None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:49.314812 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a793767-37c6-45aa-9afc-d91d31862ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:49.315727 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a793767-37c6-45aa-9afc-d91d31862ae1 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:49.316004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a793767-37c6-45aa-9afc-d91d31862ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:49.316280 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a793767-37c6-45aa-9afc-d91d31862ae1 None admin] PATH_INFO: `/v3/projects/daf205a69cab4ea8b28e786b9cb4c984` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:49.317579 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a793767-37c6-45aa-9afc-d91d31862ae1 None admin] RBAC: Authorizing `identity:get_project(project_id=daf205a69cab4ea8b28e786b9cb4c984)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:49.318961 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a793767-37c6-45aa-9afc-d91d31862ae1 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:49.320875 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 542/1066] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:10:49 2022] GET /identity/v3/projects/daf205a69cab4ea8b28e786b9cb4c984 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:49.326666 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a793767-37c6-45aa-9afc-d91d31862ae1 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:49.383767 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a72055-cc21-49fe-a6f6-242acadebc69 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:49.423150 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac77a0-af31-4610-a2a1-4ff8761087c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:49.424203 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac77a0-af31-4610-a2a1-4ff8761087c3 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:49.424493 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac77a0-af31-4610-a2a1-4ff8761087c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:49.424768 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac77a0-af31-4610-a2a1-4ff8761087c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:49.435813 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ac77a0-af31-4610-a2a1-4ff8761087c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:49.437301 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ac77a0-af31-4610-a2a1-4ff8761087c3 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:49.440729 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a793767-37c6-45aa-9afc-d91d31862ae1 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:49.455282 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-48ac77a0-af31-4610-a2a1-4ff8761087c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:49.455282 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:49.455282 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:49.455282 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:49.455282 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:49.455282 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:49.455282 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:49.455282 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:49.455282 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:49.455282 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:49.455282 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:49.455282 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:49.455282 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:49.455282 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:49.455282 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:49.456245 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:49.456245 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:49.456245 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:49.456245 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:49.456245 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:49.456245 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:49.456245 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:49.456245 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:49.456245 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:49.456245 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:49.456245 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:49.456245 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:49.456245 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:49.456245 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:49.456245 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:49.456245 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:49.456245 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:49.457191 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:49.457191 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:49.457191 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:49.457191 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:49.457191 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 525/1067] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:49.462286 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac77a0-af31-4610-a2a1-4ff8761087c3 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:49.483275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d3300a-9073-4d93-9704-96d5a27d2fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:49.483943 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3300a-9073-4d93-9704-96d5a27d2fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:49.484604 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3300a-9073-4d93-9704-96d5a27d2fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:49.484901 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3300a-9073-4d93-9704-96d5a27d2fb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:49.485764 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d3300a-9073-4d93-9704-96d5a27d2fb0 None admin] RBAC: Adding query filter params (role.id=b4c524dd7bb44e94bd3b42d86e1cdf42, scope.project.id=daf205a69cab4ea8b28e786b9cb4c984, user.id=6fe9e3df88d44547a5999968c0b3f2e5) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:49.486102 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d3300a-9073-4d93-9704-96d5a27d2fb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:49.487544 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d3300a-9073-4d93-9704-96d5a27d2fb0 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:49.493989 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 543/1068] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:10:49 2022] GET /identity/v3/role_assignments?user.id=6fe9e3df88d44547a5999968c0b3f2e5&scope.project.id=daf205a69cab4ea8b28e786b9cb4c984&role.id=b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:49.494123 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:49.578618 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe03e9c-1026-47c8-a313-2b0f9f0ea7fe None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:49.578999 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe03e9c-1026-47c8-a313-2b0f9f0ea7fe None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:49.579170 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe03e9c-1026-47c8-a313-2b0f9f0ea7fe None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:49.580985 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 544/1069] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:10:49.581705 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac77a0-af31-4610-a2a1-4ff8761087c3 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:49.586569 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1146e-28aa-4b9d-83c3-7f0c83e9000a None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:49.586897 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1146e-28aa-4b9d-83c3-7f0c83e9000a None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:49.587081 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1146e-28aa-4b9d-83c3-7f0c83e9000a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:49.608651 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-0eb1146e-28aa-4b9d-83c3-7f0c83e9000a None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:49.610369 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-0eb1146e-28aa-4b9d-83c3-7f0c83e9000a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:49.622415 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b2783-3a9e-4cf5-b0f8-574cadeb1044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:49.623119 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b2783-3a9e-4cf5-b0f8-574cadeb1044 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:49.623363 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b2783-3a9e-4cf5-b0f8-574cadeb1044 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:49.623633 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b2783-3a9e-4cf5-b0f8-574cadeb1044 None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:49.624358 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39b2783-3a9e-4cf5-b0f8-574cadeb1044 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:49.624658 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39b2783-3a9e-4cf5-b0f8-574cadeb1044 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:49.626232 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39b2783-3a9e-4cf5-b0f8-574cadeb1044 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:49.634437 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:49.646544 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 526/1070] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:10:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:49.647558 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b2783-3a9e-4cf5-b0f8-574cadeb1044 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:49.694107 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 545/1071] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:10:49.716273 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1146e-28aa-4b9d-83c3-7f0c83e9000a None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:49.750770 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:49.762269 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b2783-3a9e-4cf5-b0f8-574cadeb1044 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:49.798098 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5aa974-184f-4b3e-8bc3-869262f1e49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:49.799152 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5aa974-184f-4b3e-8bc3-869262f1e49c None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:49.799554 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5aa974-184f-4b3e-8bc3-869262f1e49c None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:49.799985 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5aa974-184f-4b3e-8bc3-869262f1e49c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:49.806920 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5aa974-184f-4b3e-8bc3-869262f1e49c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:49.808373 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5aa974-184f-4b3e-8bc3-869262f1e49c None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:49.820759 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1146e-28aa-4b9d-83c3-7f0c83e9000a None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:49.822715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-0e5aa974-184f-4b3e-8bc3-869262f1e49c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:49.822715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:49.822715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:49.822715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:49.822715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:49.822715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:49.822715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:49.822715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:49.822715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:49.822715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:49.822715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:49.822715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:49.822715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:10:49.822715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:10:49.822715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:10:49.823629 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:10:49.823629 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:49.823629 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:49.823629 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:49.823629 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:49.823629 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:49.823629 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:49.823629 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:49.823629 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:49.823629 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:49.823629 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:49.823629 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:49.823629 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:49.823629 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:49.823629 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:49.823629 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:49.823629 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:49.824532 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:10:49.824532 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:10:49.824532 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:10:49.824532 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:10:49.824532 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:10:49.824532 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:10:49.824532 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:49.824532 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:49.824532 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 527/1072] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:10:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:49.829523 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5aa974-184f-4b3e-8bc3-869262f1e49c None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:49.868495 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199e0f0-7151-4f4e-9c4e-b6edc3e39d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:49.869177 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199e0f0-7151-4f4e-9c4e-b6edc3e39d4b None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:49.869456 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199e0f0-7151-4f4e-9c4e-b6edc3e39d4b None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:49.869725 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199e0f0-7151-4f4e-9c4e-b6edc3e39d4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:49.894908 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:49.941587 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5aa974-184f-4b3e-8bc3-869262f1e49c None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:49.943286 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5199e0f0-7151-4f4e-9c4e-b6edc3e39d4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:49.944817 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5199e0f0-7151-4f4e-9c4e-b6edc3e39d4b None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:49.982938 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8152aa1f-6231-4d07-a8a1-80e70a8f34ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:49.983669 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152aa1f-6231-4d07-a8a1-80e70a8f34ec None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:49.983949 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152aa1f-6231-4d07-a8a1-80e70a8f34ec None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:49.984275 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152aa1f-6231-4d07-a8a1-80e70a8f34ec None admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:49.985098 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8152aa1f-6231-4d07-a8a1-80e70a8f34ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:49.985421 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8152aa1f-6231-4d07-a8a1-80e70a8f34ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:49.987126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8152aa1f-6231-4d07-a8a1-80e70a8f34ec None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:50.000204 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 528/1073] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 03:10:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:50.006954 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8152aa1f-6231-4d07-a8a1-80e70a8f34ec None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:50.020441 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:50.065531 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 546/1074] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:10:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 03:10:50.071474 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199e0f0-7151-4f4e-9c4e-b6edc3e39d4b None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:50.114912 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8152aa1f-6231-4d07-a8a1-80e70a8f34ec None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:50.130654 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:50.151731 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d40485-fbba-4145-bb9b-8d60e2b0cb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:50.152810 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d40485-fbba-4145-bb9b-8d60e2b0cb4b None admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:50.152934 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d40485-fbba-4145-bb9b-8d60e2b0cb4b None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:50.153195 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d40485-fbba-4145-bb9b-8d60e2b0cb4b None admin] PATH_INFO: `/v3/projects/f8398941252346468483ea0c863122f4/users/1a699988f08a4001b3b7d764f01f4fc5/roles/8c199022919d4ef2b185859d8e1dc9a9` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:50.155379 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d40485-fbba-4145-bb9b-8d60e2b0cb4b None admin] RBAC: Authorizing `identity:create_grant(project_id=f8398941252346468483ea0c863122f4, user_id=1a699988f08a4001b3b7d764f01f4fc5, role_id=8c199022919d4ef2b185859d8e1dc9a9)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:50.156985 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d40485-fbba-4145-bb9b-8d60e2b0cb4b None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:50.172586 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 529/1075] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 03:10:50 2022] PUT /identity/v3/projects/f8398941252346468483ea0c863122f4/users/1a699988f08a4001b3b7d764f01f4fc5/roles/8c199022919d4ef2b185859d8e1dc9a9 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:10:50.196108 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199e0f0-7151-4f4e-9c4e-b6edc3e39d4b None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:50.234215 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222cb7a9-e751-4ad0-b398-2735c59f6207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:50.235052 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222cb7a9-e751-4ad0-b398-2735c59f6207 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:50.235318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222cb7a9-e751-4ad0-b398-2735c59f6207 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:50.235579 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222cb7a9-e751-4ad0-b398-2735c59f6207 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:50.248320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222cb7a9-e751-4ad0-b398-2735c59f6207 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:50.249873 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222cb7a9-e751-4ad0-b398-2735c59f6207 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:50.270592 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-222cb7a9-e751-4ad0-b398-2735c59f6207 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:50.270592 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:50.270592 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:50.270592 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:50.270592 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:50.270592 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:50.270592 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:50.270592 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:50.270592 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:50.270592 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:50.270592 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:50.270592 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:50.270592 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:50.270592 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:50.270592 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:50.276792 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:50.276792 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:50.276792 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:50.276792 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:50.276792 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:50.276792 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:50.276792 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:50.276792 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:50.276792 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:50.276792 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:50.276792 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:50.276792 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:50.276792 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:50.276792 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:50.276792 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:50.276792 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:50.276792 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:50.277725 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:50.277725 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:50.277725 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:50.277725 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:50.277725 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 547/1076] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:10:50.282208 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d40485-fbba-4145-bb9b-8d60e2b0cb4b None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:50.365435 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:50.398173 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d40485-fbba-4145-bb9b-8d60e2b0cb4b None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:50.438262 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05deb664-d866-4672-a918-38c125c4f6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:50.438933 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05deb664-d866-4672-a918-38c125c4f6e1 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:50.439462 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05deb664-d866-4672-a918-38c125c4f6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:50.439698 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05deb664-d866-4672-a918-38c125c4f6e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:50.440449 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05deb664-d866-4672-a918-38c125c4f6e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:50.440749 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05deb664-d866-4672-a918-38c125c4f6e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:50.442250 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05deb664-d866-4672-a918-38c125c4f6e1 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:50.455512 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 530/1077] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:10:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:51.684735 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2a585-5ecf-4688-a8ab-46cd0a1d9c64 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:51.685323 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2a585-5ecf-4688-a8ab-46cd0a1d9c64 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:51.685323 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2a585-5ecf-4688-a8ab-46cd0a1d9c64 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:51.686693 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 548/1078] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:51.692258 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4dbbac-b9e7-4ffa-b2de-93e79671f581 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:51.692707 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4dbbac-b9e7-4ffa-b2de-93e79671f581 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:51.692824 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4dbbac-b9e7-4ffa-b2de-93e79671f581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:51.709754 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-2b4dbbac-b9e7-4ffa-b2de-93e79671f581 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:51.710943 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-2b4dbbac-b9e7-4ffa-b2de-93e79671f581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:51.779514 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 531/1079] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:10:51.802534 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f2a585-5ecf-4688-a8ab-46cd0a1d9c64 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:51.910316 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f2a585-5ecf-4688-a8ab-46cd0a1d9c64 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:51.917816 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:51.947665 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617a244-472b-4672-a2b0-4edb582ae4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:51.948382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617a244-472b-4672-a2b0-4edb582ae4fc None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:51.948660 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617a244-472b-4672-a2b0-4edb582ae4fc None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:51.948919 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c617a244-472b-4672-a2b0-4edb582ae4fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:51.959488 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c617a244-472b-4672-a2b0-4edb582ae4fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:51.967092 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-c617a244-472b-4672-a2b0-4edb582ae4fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:10:51.967092 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:51.967092 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:51.967092 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:51.967092 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:51.967092 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:51.967092 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:51.967092 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:51.967092 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:51.967092 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:51.967092 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:51.967092 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:51.967092 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:51.967092 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:51.967092 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:51.967092 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:51.968085 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:51.968085 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:51.968085 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:51.968085 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:51.968085 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:51.968085 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:51.968085 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:51.968085 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:51.968085 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:51.968085 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:51.968085 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:51.968085 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:51.968085 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:51.968085 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:51.968085 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:51.968085 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:51.968918 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:51.968918 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:51.968918 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:51.968918 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:51.968918 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:51.968918 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:51.968918 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:51.968918 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:51.968918 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:51.968918 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:51.968918 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:10:51.968918 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:51.968918 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 549/1080] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 03:10:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:51.974685 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4dbbac-b9e7-4ffa-b2de-93e79671f581 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:52.037808 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:52.083339 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4dbbac-b9e7-4ffa-b2de-93e79671f581 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:52.116569 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed278bc9-0dac-4602-8a2f-b3fab9409a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:52.117276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed278bc9-0dac-4602-8a2f-b3fab9409a9f None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:52.117554 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed278bc9-0dac-4602-8a2f-b3fab9409a9f None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:52.117827 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed278bc9-0dac-4602-8a2f-b3fab9409a9f None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:52.118577 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed278bc9-0dac-4602-8a2f-b3fab9409a9f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:52.118881 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed278bc9-0dac-4602-8a2f-b3fab9409a9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:52.120278 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed278bc9-0dac-4602-8a2f-b3fab9409a9f None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:52.129846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 532/1081] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:10:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:52.135917 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617a244-472b-4672-a2b0-4edb582ae4fc None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:52.242803 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c617a244-472b-4672-a2b0-4edb582ae4fc None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:52.278052 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb797b6-851a-44a7-a15d-a18710942cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:52.278932 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb797b6-851a-44a7-a15d-a18710942cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:52.279208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb797b6-851a-44a7-a15d-a18710942cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:52.279473 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb797b6-851a-44a7-a15d-a18710942cd4 None admin] PATH_INFO: `/v3/users/6fe9e3df88d44547a5999968c0b3f2e5` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:52.280695 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb797b6-851a-44a7-a15d-a18710942cd4 None admin] RBAC: Authorizing `identity:get_user(user_id=6fe9e3df88d44547a5999968c0b3f2e5)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:52.282175 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb797b6-851a-44a7-a15d-a18710942cd4 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:52.284268 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 550/1082] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:10:52 2022] GET /identity/v3/users/6fe9e3df88d44547a5999968c0b3f2e5 => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:52.290285 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed278bc9-0dac-4602-8a2f-b3fab9409a9f None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:52.375621 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc7847-d0e2-4a8e-97b4-809fe8ca30bc None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:52.375877 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc7847-d0e2-4a8e-97b4-809fe8ca30bc None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:52.376161 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc7847-d0e2-4a8e-97b4-809fe8ca30bc None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:52.378238 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 551/1083] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:52.383419 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd6a5fb-148b-4f85-9c00-45a55ebb0427 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:52.383696 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd6a5fb-148b-4f85-9c00-45a55ebb0427 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:52.383971 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd6a5fb-148b-4f85-9c00-45a55ebb0427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:52.401499 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-abd6a5fb-148b-4f85-9c00-45a55ebb0427 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:52.401714 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed278bc9-0dac-4602-8a2f-b3fab9409a9f None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:52.402678 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-abd6a5fb-148b-4f85-9c00-45a55ebb0427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:52.427119 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:52.432110 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:52.440057 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7fcc9-fe46-4526-8d07-7bbcf6eacfb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:52.440927 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7fcc9-fe46-4526-8d07-7bbcf6eacfb2 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:52.441232 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7fcc9-fe46-4526-8d07-7bbcf6eacfb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:52.441508 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e7fcc9-fe46-4526-8d07-7bbcf6eacfb2 None admin] PATH_INFO: `/v3/projects/daf205a69cab4ea8b28e786b9cb4c984` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:52.442668 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e7fcc9-fe46-4526-8d07-7bbcf6eacfb2 None admin] RBAC: Authorizing `identity:get_project(project_id=daf205a69cab4ea8b28e786b9cb4c984)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:52.444113 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e7fcc9-fe46-4526-8d07-7bbcf6eacfb2 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:52.445839 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 533/1084] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:10:52 2022] GET /identity/v3/projects/daf205a69cab4ea8b28e786b9cb4c984 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:52.451861 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7fcc9-fe46-4526-8d07-7bbcf6eacfb2 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:52.475362 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 552/1085] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:10:52.498956 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd6a5fb-148b-4f85-9c00-45a55ebb0427 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:52.553219 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e7fcc9-fe46-4526-8d07-7bbcf6eacfb2 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:52.587087 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54ad1e-8948-4c19-9054-1338db09ce58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:52.587990 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54ad1e-8948-4c19-9054-1338db09ce58 None admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:52.588270 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54ad1e-8948-4c19-9054-1338db09ce58 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:52.588543 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54ad1e-8948-4c19-9054-1338db09ce58 None admin] PATH_INFO: `/v3/projects/daf205a69cab4ea8b28e786b9cb4c984/users/6fe9e3df88d44547a5999968c0b3f2e5/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:52.590642 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b54ad1e-8948-4c19-9054-1338db09ce58 None admin] RBAC: Authorizing `identity:create_grant(project_id=daf205a69cab4ea8b28e786b9cb4c984, user_id=6fe9e3df88d44547a5999968c0b3f2e5, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:52.592106 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b54ad1e-8948-4c19-9054-1338db09ce58 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:52.602216 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd6a5fb-148b-4f85-9c00-45a55ebb0427 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:52.607006 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 534/1086] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 03:10:52 2022] PUT /identity/v3/projects/daf205a69cab4ea8b28e786b9cb4c984/users/6fe9e3df88d44547a5999968c0b3f2e5/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:10:52.640816 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678331c-dc84-488e-954b-bc5a83ddfb1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:52.641756 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678331c-dc84-488e-954b-bc5a83ddfb1c None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:52.642033 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678331c-dc84-488e-954b-bc5a83ddfb1c None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:52.642384 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678331c-dc84-488e-954b-bc5a83ddfb1c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:52.654667 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1678331c-dc84-488e-954b-bc5a83ddfb1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:52.663541 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-1678331c-dc84-488e-954b-bc5a83ddfb1c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:10:52.663541 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:52.663541 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:52.663541 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:52.663541 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:52.663541 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:52.663541 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:52.663541 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:52.663541 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:52.663541 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:52.663541 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:52.663541 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:52.663541 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:52.663541 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:52.663541 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:52.663541 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:52.664670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:52.664670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:52.664670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:52.664670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:52.664670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:52.664670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:52.664670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:52.664670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:52.664670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:52.664670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:52.664670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:52.664670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:52.664670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:52.664670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:52.664670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:52.664670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:52.665547 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:52.665547 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:52.665547 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:52.665547 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:52.665547 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:52.665547 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:52.665547 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:52.665547 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:52.665547 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:52.665547 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:52.665547 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 03:10:52.665547 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:52.665547 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 553/1087] 10.222.0.50 () {56 vars in 1120 bytes} [Fri May 27 03:10:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:52.671380 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54ad1e-8948-4c19-9054-1338db09ce58 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:52.732041 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:52.786739 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b54ad1e-8948-4c19-9054-1338db09ce58 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:52.822914 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900277a3-f73b-49b2-b9fe-7894e9068a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:52.823589 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900277a3-f73b-49b2-b9fe-7894e9068a75 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:52.823868 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900277a3-f73b-49b2-b9fe-7894e9068a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:52.824137 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900277a3-f73b-49b2-b9fe-7894e9068a75 None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:52.824882 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900277a3-f73b-49b2-b9fe-7894e9068a75 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:52.825218 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900277a3-f73b-49b2-b9fe-7894e9068a75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:52.826589 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900277a3-f73b-49b2-b9fe-7894e9068a75 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:52.835029 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 535/1088] 10.222.0.50 () {56 vars in 1115 bytes} [Fri May 27 03:10:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:52.840889 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678331c-dc84-488e-954b-bc5a83ddfb1c None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:52.866752 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:52.942030 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678331c-dc84-488e-954b-bc5a83ddfb1c None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:52.976573 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75007385-d770-41f8-86a3-fed4002bc34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:52.977396 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75007385-d770-41f8-86a3-fed4002bc34a None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:52.977673 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75007385-d770-41f8-86a3-fed4002bc34a None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:52.977937 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75007385-d770-41f8-86a3-fed4002bc34a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:52.988787 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75007385-d770-41f8-86a3-fed4002bc34a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:52.990269 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75007385-d770-41f8-86a3-fed4002bc34a None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:53.007134 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-75007385-d770-41f8-86a3-fed4002bc34a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:53.007134 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:53.007134 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:53.007134 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:53.007134 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:53.007134 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:53.007134 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:53.007134 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:53.007134 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:53.007134 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:53.007134 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:53.007134 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:53.007134 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:53.007134 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:53.007134 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:53.008078 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:53.008078 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:53.008078 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:53.008078 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:53.008078 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:53.008078 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:53.008078 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:53.008078 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:53.008078 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:53.008078 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:53.008078 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:53.008078 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:53.008078 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:53.008078 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:53.008078 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:53.008078 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:53.008078 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:53.008991 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:53.008991 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:53.008991 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:53.008991 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:53.008991 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 554/1089] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:53.014233 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900277a3-f73b-49b2-b9fe-7894e9068a75 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:53.119378 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900277a3-f73b-49b2-b9fe-7894e9068a75 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:53.132982 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:53.155567 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aac967-f08d-4d81-8dde-720d5719e1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:53.156235 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aac967-f08d-4d81-8dde-720d5719e1fd None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:53.156513 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aac967-f08d-4d81-8dde-720d5719e1fd None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:53.156790 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aac967-f08d-4d81-8dde-720d5719e1fd None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:53.157547 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aac967-f08d-4d81-8dde-720d5719e1fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:53.158358 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aac967-f08d-4d81-8dde-720d5719e1fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:53.159769 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aac967-f08d-4d81-8dde-720d5719e1fd None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:53.171658 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 536/1090] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:10:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:53.177408 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75007385-d770-41f8-86a3-fed4002bc34a None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:53.259299 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:53.283888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75007385-d770-41f8-86a3-fed4002bc34a None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:53.319232 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a245847-6519-4980-b5eb-0c24bae4035c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:53.320041 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a245847-6519-4980-b5eb-0c24bae4035c None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:53.320321 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a245847-6519-4980-b5eb-0c24bae4035c None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:53.320596 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a245847-6519-4980-b5eb-0c24bae4035c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:53.328851 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a245847-6519-4980-b5eb-0c24bae4035c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:53.330344 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a245847-6519-4980-b5eb-0c24bae4035c None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:53.344484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-6a245847-6519-4980-b5eb-0c24bae4035c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 03:10:53.344484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:53.344484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:53.344484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:53.344484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:53.344484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:53.344484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:53.344484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:53.344484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:53.344484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:53.344484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:53.344484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:53.344484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:10:53.344484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:10:53.344484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:10:53.344484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:10:53.345585 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:53.345585 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:53.345585 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:10:53.345585 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:53.345585 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:10:53.345585 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:10:53.345585 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:53.345585 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:53.345585 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:53.345585 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:53.345585 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:53.345585 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:53.345585 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:53.345585 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:53.345585 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:53.345585 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:53.345585 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:53.345585 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:53.346533 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:53.346533 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:53.346533 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:10:53.346533 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:10:53.346533 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:10:53.346533 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:10:53.346533 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:10:53.346533 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:10:53.346533 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 03:10:53.346533 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:53.346533 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 555/1091] 10.222.0.50 () {56 vars in 1093 bytes} [Fri May 27 03:10:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:53.351787 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aac967-f08d-4d81-8dde-720d5719e1fd None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:53.456010 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aac967-f08d-4d81-8dde-720d5719e1fd None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:53.491023 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6764ac-032d-4a9b-89ed-a4415445f0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:53.491700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6764ac-032d-4a9b-89ed-a4415445f0e3 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:53.491979 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6764ac-032d-4a9b-89ed-a4415445f0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:53.492251 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6764ac-032d-4a9b-89ed-a4415445f0e3 None admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:53.493122 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6764ac-032d-4a9b-89ed-a4415445f0e3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:53.493431 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6764ac-032d-4a9b-89ed-a4415445f0e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:53.494875 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6764ac-032d-4a9b-89ed-a4415445f0e3 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:53.512686 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 537/1092] 10.222.0.50 () {56 vars in 1133 bytes} [Fri May 27 03:10:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:53.518518 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a245847-6519-4980-b5eb-0c24bae4035c None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:53.617448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a245847-6519-4980-b5eb-0c24bae4035c None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:53.652193 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ec6ae1-818c-47ea-bcee-e8a6b5552ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:53.652835 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec6ae1-818c-47ea-bcee-e8a6b5552ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:53.653139 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec6ae1-818c-47ea-bcee-e8a6b5552ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:53.653412 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec6ae1-818c-47ea-bcee-e8a6b5552ae6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:53.656976 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:53.724710 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ec6ae1-818c-47ea-bcee-e8a6b5552ae6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:53.726099 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ec6ae1-818c-47ea-bcee-e8a6b5552ae6 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:53.831262 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 556/1093] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:10:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:10:53.837386 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6764ac-032d-4a9b-89ed-a4415445f0e3 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:53.868459 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:53.937449 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6764ac-032d-4a9b-89ed-a4415445f0e3 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:53.972011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083d9b73-5955-47d4-8247-73b5a27d50ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:53.972798 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d9b73-5955-47d4-8247-73b5a27d50ee None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:53.973104 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d9b73-5955-47d4-8247-73b5a27d50ee None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:53.973369 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d9b73-5955-47d4-8247-73b5a27d50ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:53.983540 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083d9b73-5955-47d4-8247-73b5a27d50ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:53.984946 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083d9b73-5955-47d4-8247-73b5a27d50ee None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:54.003107 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-083d9b73-5955-47d4-8247-73b5a27d50ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:54.003107 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:54.003107 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:54.003107 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:54.003107 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:54.003107 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:54.003107 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:54.003107 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:54.003107 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:54.003107 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:54.003107 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:54.003107 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:54.003107 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:10:54.003107 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:10:54.003107 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:10:54.004126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:10:54.004126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:54.004126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:54.004126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:54.004126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:54.004126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:54.004126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:54.004126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:54.004126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:54.004126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:54.004126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:54.004126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:54.004126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:54.004126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:54.004126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:54.004126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:54.004126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:54.005199 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:10:54.005199 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:10:54.005199 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:10:54.005199 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:54.005199 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 538/1094] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:54.010203 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ec6ae1-818c-47ea-bcee-e8a6b5552ae6 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:54.053068 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:54.122741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ec6ae1-818c-47ea-bcee-e8a6b5552ae6 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:54.157337 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdce1160-aa17-4cdb-8b6f-fe5f17ca52a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:54.158382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce1160-aa17-4cdb-8b6f-fe5f17ca52a4 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:54.158767 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce1160-aa17-4cdb-8b6f-fe5f17ca52a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:54.158897 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce1160-aa17-4cdb-8b6f-fe5f17ca52a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:54.159634 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdce1160-aa17-4cdb-8b6f-fe5f17ca52a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:54.159934 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdce1160-aa17-4cdb-8b6f-fe5f17ca52a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:54.161332 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdce1160-aa17-4cdb-8b6f-fe5f17ca52a4 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:54.172643 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 557/1095] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:10:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:54.178487 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083d9b73-5955-47d4-8247-73b5a27d50ee None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:54.261242 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:54.282203 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083d9b73-5955-47d4-8247-73b5a27d50ee None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:54.318858 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fa3837-78e2-46cb-8332-c851aa55d3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:54.319660 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa3837-78e2-46cb-8332-c851aa55d3b4 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:54.319933 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa3837-78e2-46cb-8332-c851aa55d3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:54.320225 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa3837-78e2-46cb-8332-c851aa55d3b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:54.327305 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fa3837-78e2-46cb-8332-c851aa55d3b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:54.328733 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fa3837-78e2-46cb-8332-c851aa55d3b4 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:54.342900 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-04fa3837-78e2-46cb-8332-c851aa55d3b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:54.342900 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:54.342900 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:54.342900 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:54.342900 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:54.342900 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:54.342900 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:54.342900 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:54.342900 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:54.342900 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:54.342900 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:54.342900 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:54.342900 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:10:54.342900 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:10:54.342900 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:10:54.343936 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:10:54.343936 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:54.343936 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:54.343936 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:54.343936 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:54.343936 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:54.343936 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:54.343936 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:54.343936 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:10:54.343936 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:54.343936 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:54.343936 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:54.343936 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:54.343936 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:54.343936 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:54.343936 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:54.343936 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:54.350715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:10:54.350715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:10:54.350715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:10:54.350715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:10:54.350715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:10:54.350715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:10:54.350715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:10:54.350715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:10:54.350715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 539/1096] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:10:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:54.351434 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdce1160-aa17-4cdb-8b6f-fe5f17ca52a4 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:54.456390 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:54.463215 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdce1160-aa17-4cdb-8b6f-fe5f17ca52a4 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:54.500165 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121f04f6-3cf1-4b5d-aaef-83580e38b1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:54.500831 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f04f6-3cf1-4b5d-aaef-83580e38b1b0 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:54.501142 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f04f6-3cf1-4b5d-aaef-83580e38b1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:54.501463 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121f04f6-3cf1-4b5d-aaef-83580e38b1b0 None admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:54.502484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121f04f6-3cf1-4b5d-aaef-83580e38b1b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:54.502790 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121f04f6-3cf1-4b5d-aaef-83580e38b1b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:54.504236 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121f04f6-3cf1-4b5d-aaef-83580e38b1b0 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:54.516742 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 558/1097] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 03:10:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:54.522793 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fa3837-78e2-46cb-8332-c851aa55d3b4 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:54.629807 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fa3837-78e2-46cb-8332-c851aa55d3b4 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:54.667412 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd8065-1212-49db-b965-0b60f155ff89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:54.668097 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd8065-1212-49db-b965-0b60f155ff89 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:54.668374 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd8065-1212-49db-b965-0b60f155ff89 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:54.668644 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd8065-1212-49db-b965-0b60f155ff89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:54.669525 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd8065-1212-49db-b965-0b60f155ff89 None admin] RBAC: Adding query filter params (role.id=8c199022919d4ef2b185859d8e1dc9a9, scope.project.id=f8398941252346468483ea0c863122f4, user.id=1a699988f08a4001b3b7d764f01f4fc5) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:54.669832 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd8065-1212-49db-b965-0b60f155ff89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:54.671301 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd8065-1212-49db-b965-0b60f155ff89 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:54.689095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 540/1098] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:10:54 2022] GET /identity/v3/role_assignments?user.id=1a699988f08a4001b3b7d764f01f4fc5&scope.project.id=f8398941252346468483ea0c863122f4&role.id=8c199022919d4ef2b185859d8e1dc9a9 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:54.857734 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541d4dd-2b3f-4324-806b-a5f970b8ca9b None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:54.857938 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541d4dd-2b3f-4324-806b-a5f970b8ca9b None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:54.858198 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541d4dd-2b3f-4324-806b-a5f970b8ca9b None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:54.859630 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 559/1099] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:54.865702 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eacdce6-a84e-4b3b-b072-2af71e5f3309 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:54.866179 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eacdce6-a84e-4b3b-b072-2af71e5f3309 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:54.866272 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eacdce6-a84e-4b3b-b072-2af71e5f3309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:54.881171 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:54.887090 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-0eacdce6-a84e-4b3b-b072-2af71e5f3309 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:54.888319 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-0eacdce6-a84e-4b3b-b072-2af71e5f3309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:54.956451 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 541/1100] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:10:54.980576 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541d4dd-2b3f-4324-806b-a5f970b8ca9b None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:55.088135 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541d4dd-2b3f-4324-806b-a5f970b8ca9b None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:55.123187 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214cf2b-a028-4997-85b8-0979dc585018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:55.123914 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214cf2b-a028-4997-85b8-0979dc585018 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:55.124194 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214cf2b-a028-4997-85b8-0979dc585018 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:55.124467 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214cf2b-a028-4997-85b8-0979dc585018 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:55.135269 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9214cf2b-a028-4997-85b8-0979dc585018 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:10:55.142923 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-9214cf2b-a028-4997-85b8-0979dc585018 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:10:55.142923 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:10:55.142923 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:10:55.142923 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:10:55.142923 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:10:55.142923 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:10:55.142923 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:10:55.142923 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:10:55.142923 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:10:55.142923 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:10:55.142923 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:10:55.142923 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:10:55.142923 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:10:55.142923 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:10:55.142923 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:10:55.142923 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:10:55.144023 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:10:55.144023 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:10:55.144023 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:10:55.144023 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:10:55.144023 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:10:55.144023 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:10:55.144023 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:10:55.144023 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:10:55.144023 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:10:55.144023 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:10:55.144023 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:10:55.144023 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:10:55.144023 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:10:55.144023 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:10:55.144023 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:10:55.144023 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:10:55.144859 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:10:55.144859 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:10:55.144859 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:10:55.144859 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:10:55.144859 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:10:55.144859 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:10:55.144859 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:10:55.144859 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:10:55.144859 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:10:55.144859 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:10:55.144859 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:10:55.144859 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:10:55.144859 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 560/1101] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 03:10:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:10:55.151092 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eacdce6-a84e-4b3b-b072-2af71e5f3309 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:55.259864 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eacdce6-a84e-4b3b-b072-2af71e5f3309 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:55.301205 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc75cb-6357-4d6f-926a-0963d61238c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:55.301900 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc75cb-6357-4d6f-926a-0963d61238c9 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:55.302184 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc75cb-6357-4d6f-926a-0963d61238c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:55.302459 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc75cb-6357-4d6f-926a-0963d61238c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:55.303224 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcc75cb-6357-4d6f-926a-0963d61238c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:55.303535 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcc75cb-6357-4d6f-926a-0963d61238c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:55.304923 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcc75cb-6357-4d6f-926a-0963d61238c9 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:55.307690 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:55.315072 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 542/1102] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:10:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:55.320887 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214cf2b-a028-4997-85b8-0979dc585018 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:55.340585 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:55.425262 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214cf2b-a028-4997-85b8-0979dc585018 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:55.460872 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c257201f-2673-4c81-92e7-0fc66cce72ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:55.461803 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257201f-2673-4c81-92e7-0fc66cce72ba None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:55.462081 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257201f-2673-4c81-92e7-0fc66cce72ba None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:55.462352 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c257201f-2673-4c81-92e7-0fc66cce72ba None admin] PATH_INFO: `/v3/users/6fe9e3df88d44547a5999968c0b3f2e5` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:55.463539 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c257201f-2673-4c81-92e7-0fc66cce72ba None admin] RBAC: Authorizing `identity:get_user(user_id=6fe9e3df88d44547a5999968c0b3f2e5)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:55.465022 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c257201f-2673-4c81-92e7-0fc66cce72ba None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:55.466678 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 561/1103] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:10:55 2022] GET /identity/v3/users/6fe9e3df88d44547a5999968c0b3f2e5 => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:55.472897 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc75cb-6357-4d6f-926a-0963d61238c9 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:55.578634 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcc75cb-6357-4d6f-926a-0963d61238c9 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:55.612087 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b9c55-e3a0-42c5-974c-79dd47d1e32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:55.612735 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b9c55-e3a0-42c5-974c-79dd47d1e32c None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:55.613046 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b9c55-e3a0-42c5-974c-79dd47d1e32c None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:55.613313 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b9c55-e3a0-42c5-974c-79dd47d1e32c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:55.681556 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:55.685651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b9c55-e3a0-42c5-974c-79dd47d1e32c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:55.687510 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b9c55-e3a0-42c5-974c-79dd47d1e32c None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:55.799384 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 543/1104] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:10:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:10:55.805528 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c257201f-2673-4c81-92e7-0fc66cce72ba None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:55.899410 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:55.922943 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c257201f-2673-4c81-92e7-0fc66cce72ba None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:55.962163 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0fd55-7aa6-4518-92dd-ab9984b3b887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:55.962982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0fd55-7aa6-4518-92dd-ab9984b3b887 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:55.963252 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0fd55-7aa6-4518-92dd-ab9984b3b887 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:55.963525 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0fd55-7aa6-4518-92dd-ab9984b3b887 None admin] PATH_INFO: `/v3/projects/daf205a69cab4ea8b28e786b9cb4c984` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:55.964753 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b0fd55-7aa6-4518-92dd-ab9984b3b887 None admin] RBAC: Authorizing `identity:get_project(project_id=daf205a69cab4ea8b28e786b9cb4c984)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:55.966242 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b0fd55-7aa6-4518-92dd-ab9984b3b887 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:55.967956 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 562/1105] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:10:55 2022] GET /identity/v3/projects/daf205a69cab4ea8b28e786b9cb4c984 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:55.974520 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b9c55-e3a0-42c5-974c-79dd47d1e32c None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:56.078729 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:56.082693 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b9c55-e3a0-42c5-974c-79dd47d1e32c None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:56.120940 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a162620a-4eb5-4de9-a198-93c07b0b4395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:56.121618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a162620a-4eb5-4de9-a198-93c07b0b4395 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:56.121897 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a162620a-4eb5-4de9-a198-93c07b0b4395 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:56.122168 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a162620a-4eb5-4de9-a198-93c07b0b4395 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:56.122999 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a162620a-4eb5-4de9-a198-93c07b0b4395 None admin] RBAC: Adding query filter params (role.id=b4c524dd7bb44e94bd3b42d86e1cdf42, scope.project.id=daf205a69cab4ea8b28e786b9cb4c984, user.id=6fe9e3df88d44547a5999968c0b3f2e5) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:10:56.123308 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a162620a-4eb5-4de9-a198-93c07b0b4395 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:56.124744 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a162620a-4eb5-4de9-a198-93c07b0b4395 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:56.131321 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 544/1106] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:10:55 2022] GET /identity/v3/role_assignments?user.id=6fe9e3df88d44547a5999968c0b3f2e5&scope.project.id=daf205a69cab4ea8b28e786b9cb4c984&role.id=b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:10:58.367189 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e905714-0d97-4b09-ab83-79327643c17e None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:58.368066 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e905714-0d97-4b09-ab83-79327643c17e None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:58.368066 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e905714-0d97-4b09-ab83-79327643c17e None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:58.370583 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 563/1107] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:10:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:10:58.379465 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299c9fc-ca23-4b15-805a-9b034591f30b None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:58.379893 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299c9fc-ca23-4b15-805a-9b034591f30b None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:58.380061 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299c9fc-ca23-4b15-805a-9b034591f30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:58.407764 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-9299c9fc-ca23-4b15-805a-9b034591f30b None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:10:58.409436 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-9299c9fc-ca23-4b15-805a-9b034591f30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:10:58.479769 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 545/1108] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:10:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:10:58.504513 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e905714-0d97-4b09-ab83-79327643c17e None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:58.623153 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e905714-0d97-4b09-ab83-79327643c17e None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:58.666310 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f771e6c2-dbc1-4f08-8dd2-ee0f13efe101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:58.667102 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771e6c2-dbc1-4f08-8dd2-ee0f13efe101 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:58.667382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771e6c2-dbc1-4f08-8dd2-ee0f13efe101 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:58.667652 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f771e6c2-dbc1-4f08-8dd2-ee0f13efe101 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:58.668837 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f771e6c2-dbc1-4f08-8dd2-ee0f13efe101 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:58.670333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f771e6c2-dbc1-4f08-8dd2-ee0f13efe101 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:58.671958 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 564/1109] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:10:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:10:58.678670 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299c9fc-ca23-4b15-805a-9b034591f30b None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:10:58.786375 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:10:58.807916 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299c9fc-ca23-4b15-805a-9b034591f30b None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:10:58.848420 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654fb469-12bc-4e0e-a512-5552d62a9dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:10:58.849122 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654fb469-12bc-4e0e-a512-5552d62a9dc2 None admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:10:58.849399 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654fb469-12bc-4e0e-a512-5552d62a9dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:10:58.849670 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654fb469-12bc-4e0e-a512-5552d62a9dc2 None admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:10:58.850377 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654fb469-12bc-4e0e-a512-5552d62a9dc2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:10:58.851837 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654fb469-12bc-4e0e-a512-5552d62a9dc2 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:10:58.879445 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 546/1110] 10.222.0.50 () {60 vars in 1132 bytes} [Fri May 27 03:10:58 2022] POST /identity/v3/users => generated 286 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:11:01.084076 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d9a6d5-f1e0-48ad-a1a1-d53d39af7d3e None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:01.084682 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d9a6d5-f1e0-48ad-a1a1-d53d39af7d3e None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:01.084682 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d9a6d5-f1e0-48ad-a1a1-d53d39af7d3e None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:01.086018 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 565/1111] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:11:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:01.092453 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f0100-5a8b-4b18-929b-dac5abb56ec7 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:01.093216 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f0100-5a8b-4b18-929b-dac5abb56ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:01.093384 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f0100-5a8b-4b18-929b-dac5abb56ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:01.118367 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-b40f0100-5a8b-4b18-929b-dac5abb56ec7 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:01.119537 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-b40f0100-5a8b-4b18-929b-dac5abb56ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:01.190989 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 547/1112] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:11:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:01.219334 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d9a6d5-f1e0-48ad-a1a1-d53d39af7d3e None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:01.244492 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:01.336976 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d9a6d5-f1e0-48ad-a1a1-d53d39af7d3e None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:01.371697 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b871ade-5fa2-40d5-806c-63001b913c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:01.372563 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b871ade-5fa2-40d5-806c-63001b913c67 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:01.372848 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b871ade-5fa2-40d5-806c-63001b913c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:01.373151 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b871ade-5fa2-40d5-806c-63001b913c67 None admin] PATH_INFO: `/v3/roles/9e63b73b3ce24bb3ad214ecdb6132696` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:01.374737 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b871ade-5fa2-40d5-806c-63001b913c67 None admin] RBAC: Authorizing `identity:get_role(role_id=9e63b73b3ce24bb3ad214ecdb6132696)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:01.376127 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b871ade-5fa2-40d5-806c-63001b913c67 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:01.377627 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 566/1113] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:11:01 2022] GET /identity/v3/roles/9e63b73b3ce24bb3ad214ecdb6132696 => generated 223 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:01.386937 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40f0100-5a8b-4b18-929b-dac5abb56ec7 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:01.478625 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:01.511860 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40f0100-5a8b-4b18-929b-dac5abb56ec7 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:01.551459 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17de9391-678f-44f8-86e4-0db3ca7e2ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:01.552334 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17de9391-678f-44f8-86e4-0db3ca7e2ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:01.552609 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17de9391-678f-44f8-86e4-0db3ca7e2ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:01.552881 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17de9391-678f-44f8-86e4-0db3ca7e2ec1 None admin] PATH_INFO: `/v3/users/36fbe332d07344649d73fec27d1df8e7` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:01.567357 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17de9391-678f-44f8-86e4-0db3ca7e2ec1 None admin] RBAC: Authorizing `identity:get_user(user_id=36fbe332d07344649d73fec27d1df8e7)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:01.568821 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17de9391-678f-44f8-86e4-0db3ca7e2ec1 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:01.570506 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 548/1114] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:11:01 2022] GET /identity/v3/users/36fbe332d07344649d73fec27d1df8e7 => generated 286 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:01.577350 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b871ade-5fa2-40d5-806c-63001b913c67 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:01.679302 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:01.693086 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b871ade-5fa2-40d5-806c-63001b913c67 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:01.729521 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a286d1-caba-45e5-b63c-1b9f51fc14f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:01.730174 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a286d1-caba-45e5-b63c-1b9f51fc14f9 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:01.730451 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a286d1-caba-45e5-b63c-1b9f51fc14f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:01.730723 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a286d1-caba-45e5-b63c-1b9f51fc14f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:01.798190 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a286d1-caba-45e5-b63c-1b9f51fc14f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:01.799743 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a286d1-caba-45e5-b63c-1b9f51fc14f9 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:01.886394 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:01.900000 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 567/1115] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:11:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:11:01.910307 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17de9391-678f-44f8-86e4-0db3ca7e2ec1 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:02.022799 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17de9391-678f-44f8-86e4-0db3ca7e2ec1 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:02.059977 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f348514-fdfc-4636-aa45-932b91f59e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:02.060829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f348514-fdfc-4636-aa45-932b91f59e74 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:02.061136 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f348514-fdfc-4636-aa45-932b91f59e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:02.061404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f348514-fdfc-4636-aa45-932b91f59e74 None admin] PATH_INFO: `/v3/projects/daf205a69cab4ea8b28e786b9cb4c984` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:02.062687 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f348514-fdfc-4636-aa45-932b91f59e74 None admin] RBAC: Authorizing `identity:get_project(project_id=daf205a69cab4ea8b28e786b9cb4c984)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:02.064168 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f348514-fdfc-4636-aa45-932b91f59e74 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:02.065878 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 549/1116] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:11:01 2022] GET /identity/v3/projects/daf205a69cab4ea8b28e786b9cb4c984 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:11:02.072240 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a286d1-caba-45e5-b63c-1b9f51fc14f9 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:02.171846 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a286d1-caba-45e5-b63c-1b9f51fc14f9 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:02.207184 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10755cb-245f-4564-937c-8ed9f991c1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:02.207829 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10755cb-245f-4564-937c-8ed9f991c1c1 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:02.208108 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10755cb-245f-4564-937c-8ed9f991c1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:02.208382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10755cb-245f-4564-937c-8ed9f991c1c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:02.209234 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10755cb-245f-4564-937c-8ed9f991c1c1 None admin] RBAC: Adding query filter params (role.id=9e63b73b3ce24bb3ad214ecdb6132696, scope.project.id=daf205a69cab4ea8b28e786b9cb4c984, user.id=36fbe332d07344649d73fec27d1df8e7) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:02.209543 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10755cb-245f-4564-937c-8ed9f991c1c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:02.210979 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10755cb-245f-4564-937c-8ed9f991c1c1 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:02.216453 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 568/1117] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:11:02 2022] GET /identity/v3/role_assignments?user.id=36fbe332d07344649d73fec27d1df8e7&scope.project.id=daf205a69cab4ea8b28e786b9cb4c984&role.id=9e63b73b3ce24bb3ad214ecdb6132696 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:04.409353 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469063e0-71d4-4074-b4e1-707a85ad0f5a None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:04.409898 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469063e0-71d4-4074-b4e1-707a85ad0f5a None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:04.409898 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469063e0-71d4-4074-b4e1-707a85ad0f5a None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:04.411294 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 550/1118] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:11:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:04.417862 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2716387-5aef-447b-bce4-b06803ff36dd None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:04.418608 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2716387-5aef-447b-bce4-b06803ff36dd None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:04.418737 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2716387-5aef-447b-bce4-b06803ff36dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:04.445470 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-e2716387-5aef-447b-bce4-b06803ff36dd None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:04.446680 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-e2716387-5aef-447b-bce4-b06803ff36dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:04.502922 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:04.524879 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 569/1119] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:11:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:04.548474 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469063e0-71d4-4074-b4e1-707a85ad0f5a None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:04.552922 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:04.672368 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469063e0-71d4-4074-b4e1-707a85ad0f5a None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:04.715939 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef095f-dcce-4ceb-b05e-b5a4c71d7d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:04.717214 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef095f-dcce-4ceb-b05e-b5a4c71d7d7d None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:04.717581 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef095f-dcce-4ceb-b05e-b5a4c71d7d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:04.717960 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef095f-dcce-4ceb-b05e-b5a4c71d7d7d None admin] PATH_INFO: `/v3/roles/9e63b73b3ce24bb3ad214ecdb6132696` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:04.720517 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef095f-dcce-4ceb-b05e-b5a4c71d7d7d None admin] RBAC: Authorizing `identity:get_role(role_id=9e63b73b3ce24bb3ad214ecdb6132696)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:04.722847 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef095f-dcce-4ceb-b05e-b5a4c71d7d7d None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:04.725000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 551/1120] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:11:04 2022] GET /identity/v3/roles/9e63b73b3ce24bb3ad214ecdb6132696 => generated 223 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:04.734430 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2716387-5aef-447b-bce4-b06803ff36dd None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:04.850600 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2716387-5aef-447b-bce4-b06803ff36dd None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:04.886207 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f9e2d-e8da-4ee6-8e62-2b6a34c56d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:04.887074 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f9e2d-e8da-4ee6-8e62-2b6a34c56d76 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:04.887344 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f9e2d-e8da-4ee6-8e62-2b6a34c56d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:04.887613 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f9e2d-e8da-4ee6-8e62-2b6a34c56d76 None admin] PATH_INFO: `/v3/users/36fbe332d07344649d73fec27d1df8e7` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:04.888914 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f9e2d-e8da-4ee6-8e62-2b6a34c56d76 None admin] RBAC: Authorizing `identity:get_user(user_id=36fbe332d07344649d73fec27d1df8e7)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:04.890391 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f9e2d-e8da-4ee6-8e62-2b6a34c56d76 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:04.892085 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 570/1121] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:11:04 2022] GET /identity/v3/users/36fbe332d07344649d73fec27d1df8e7 => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:04.898408 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef095f-dcce-4ceb-b05e-b5a4c71d7d7d None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:04.978695 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:05.016005 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef095f-dcce-4ceb-b05e-b5a4c71d7d7d None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:05.051466 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51398e-0d31-4860-aff0-b8addd2a14a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:05.052310 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51398e-0d31-4860-aff0-b8addd2a14a5 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:05.052583 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51398e-0d31-4860-aff0-b8addd2a14a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:05.052851 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51398e-0d31-4860-aff0-b8addd2a14a5 None admin] PATH_INFO: `/v3/projects/daf205a69cab4ea8b28e786b9cb4c984` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:05.054075 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c51398e-0d31-4860-aff0-b8addd2a14a5 None admin] RBAC: Authorizing `identity:get_project(project_id=daf205a69cab4ea8b28e786b9cb4c984)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:05.055521 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c51398e-0d31-4860-aff0-b8addd2a14a5 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:05.057489 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 552/1122] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:11:04 2022] GET /identity/v3/projects/daf205a69cab4ea8b28e786b9cb4c984 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:05.063582 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f9e2d-e8da-4ee6-8e62-2b6a34c56d76 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:05.120710 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:05.172864 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f9e2d-e8da-4ee6-8e62-2b6a34c56d76 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:05.207254 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c35191-b788-4337-a49d-f71d1abcdf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:05.208125 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c35191-b788-4337-a49d-f71d1abcdf8e None admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:05.208395 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c35191-b788-4337-a49d-f71d1abcdf8e None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:05.208659 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c35191-b788-4337-a49d-f71d1abcdf8e None admin] PATH_INFO: `/v3/projects/daf205a69cab4ea8b28e786b9cb4c984/users/36fbe332d07344649d73fec27d1df8e7/roles/9e63b73b3ce24bb3ad214ecdb6132696` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:05.210986 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c35191-b788-4337-a49d-f71d1abcdf8e None admin] RBAC: Authorizing `identity:create_grant(project_id=daf205a69cab4ea8b28e786b9cb4c984, user_id=36fbe332d07344649d73fec27d1df8e7, role_id=9e63b73b3ce24bb3ad214ecdb6132696)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:05.212578 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c35191-b788-4337-a49d-f71d1abcdf8e None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:05.228733 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 571/1123] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 03:11:05 2022] PUT /identity/v3/projects/daf205a69cab4ea8b28e786b9cb4c984/users/36fbe332d07344649d73fec27d1df8e7/roles/9e63b73b3ce24bb3ad214ecdb6132696 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:11:07.411964 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ead002-6710-434d-b4ea-e11187d7df7d None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:07.412837 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ead002-6710-434d-b4ea-e11187d7df7d None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:07.412837 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ead002-6710-434d-b4ea-e11187d7df7d None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:07.415097 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 553/1124] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:11:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:07.423962 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95510b66-7045-46af-9285-0755bac081b3 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:07.424405 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95510b66-7045-46af-9285-0755bac081b3 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:07.424569 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95510b66-7045-46af-9285-0755bac081b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:07.447237 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-95510b66-7045-46af-9285-0755bac081b3 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:07.448435 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-95510b66-7045-46af-9285-0755bac081b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:07.519537 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 572/1125] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:11:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:07.543818 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ead002-6710-434d-b4ea-e11187d7df7d None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:07.652855 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ead002-6710-434d-b4ea-e11187d7df7d None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:07.689245 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b2472c-f0b5-4154-bc84-1a7f20eaa694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:07.690255 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2472c-f0b5-4154-bc84-1a7f20eaa694 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:07.690666 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2472c-f0b5-4154-bc84-1a7f20eaa694 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:07.690944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2472c-f0b5-4154-bc84-1a7f20eaa694 None admin] PATH_INFO: `/v3/roles/9e63b73b3ce24bb3ad214ecdb6132696` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:07.692512 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b2472c-f0b5-4154-bc84-1a7f20eaa694 None admin] RBAC: Authorizing `identity:get_role(role_id=9e63b73b3ce24bb3ad214ecdb6132696)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:07.693930 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b2472c-f0b5-4154-bc84-1a7f20eaa694 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:07.695289 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 554/1126] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:11:07 2022] GET /identity/v3/roles/9e63b73b3ce24bb3ad214ecdb6132696 => generated 223 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:07.702089 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95510b66-7045-46af-9285-0755bac081b3 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:07.757570 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:07.818939 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95510b66-7045-46af-9285-0755bac081b3 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:07.857447 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e3e98e-a939-407b-8056-f3b9aa8c8498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:07.858302 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e3e98e-a939-407b-8056-f3b9aa8c8498 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:07.858574 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e3e98e-a939-407b-8056-f3b9aa8c8498 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:07.858845 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e3e98e-a939-407b-8056-f3b9aa8c8498 None admin] PATH_INFO: `/v3/users/36fbe332d07344649d73fec27d1df8e7` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:07.860057 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e3e98e-a939-407b-8056-f3b9aa8c8498 None admin] RBAC: Authorizing `identity:get_user(user_id=36fbe332d07344649d73fec27d1df8e7)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:07.861535 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e3e98e-a939-407b-8056-f3b9aa8c8498 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:07.863215 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 573/1127] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:11:07 2022] GET /identity/v3/users/36fbe332d07344649d73fec27d1df8e7 => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:07.872599 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b2472c-f0b5-4154-bc84-1a7f20eaa694 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:07.891211 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:07.986538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b2472c-f0b5-4154-bc84-1a7f20eaa694 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:08.022824 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916110fa-7f9d-4ecc-8ece-4a5552f29627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:08.023466 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916110fa-7f9d-4ecc-8ece-4a5552f29627 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:08.023747 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916110fa-7f9d-4ecc-8ece-4a5552f29627 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:08.024020 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916110fa-7f9d-4ecc-8ece-4a5552f29627 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:08.095758 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916110fa-7f9d-4ecc-8ece-4a5552f29627 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:08.097563 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916110fa-7f9d-4ecc-8ece-4a5552f29627 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:08.107455 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:08.209351 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 555/1128] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:11:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:11:08.219148 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e3e98e-a939-407b-8056-f3b9aa8c8498 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:08.328434 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e3e98e-a939-407b-8056-f3b9aa8c8498 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:08.338188 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:08.367161 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d014f36a-bf3f-4e9e-bbd5-de9491b44b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:08.368024 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d014f36a-bf3f-4e9e-bbd5-de9491b44b6b None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:08.368287 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d014f36a-bf3f-4e9e-bbd5-de9491b44b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:08.368548 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d014f36a-bf3f-4e9e-bbd5-de9491b44b6b None admin] PATH_INFO: `/v3/projects/daf205a69cab4ea8b28e786b9cb4c984` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:08.369894 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d014f36a-bf3f-4e9e-bbd5-de9491b44b6b None admin] RBAC: Authorizing `identity:get_project(project_id=daf205a69cab4ea8b28e786b9cb4c984)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:08.371359 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d014f36a-bf3f-4e9e-bbd5-de9491b44b6b None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:08.373093 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 574/1129] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:11:08 2022] GET /identity/v3/projects/daf205a69cab4ea8b28e786b9cb4c984 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:08.379700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916110fa-7f9d-4ecc-8ece-4a5552f29627 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:08.491613 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916110fa-7f9d-4ecc-8ece-4a5552f29627 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:08.500868 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:08.528819 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c9c95-0413-4fd0-903d-a0fbc0365e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:08.529524 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c9c95-0413-4fd0-903d-a0fbc0365e89 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:08.529806 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c9c95-0413-4fd0-903d-a0fbc0365e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:08.530082 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c9c95-0413-4fd0-903d-a0fbc0365e89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:08.530907 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c9c95-0413-4fd0-903d-a0fbc0365e89 None admin] RBAC: Adding query filter params (role.id=9e63b73b3ce24bb3ad214ecdb6132696, scope.project.id=daf205a69cab4ea8b28e786b9cb4c984, user.id=36fbe332d07344649d73fec27d1df8e7) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:08.531218 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c9c95-0413-4fd0-903d-a0fbc0365e89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:08.532997 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c9c95-0413-4fd0-903d-a0fbc0365e89 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:08.539311 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 556/1130] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:11:08 2022] GET /identity/v3/role_assignments?user.id=36fbe332d07344649d73fec27d1df8e7&scope.project.id=daf205a69cab4ea8b28e786b9cb4c984&role.id=9e63b73b3ce24bb3ad214ecdb6132696 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:10.819555 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb4780-4c0a-4eb5-897f-78999825c018 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:10.820113 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb4780-4c0a-4eb5-897f-78999825c018 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:10.820113 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbb4780-4c0a-4eb5-897f-78999825c018 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:10.822295 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 575/1131] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:11:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:10.831152 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45f392-74b3-4257-997b-9524ad774b1a None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:10.831535 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45f392-74b3-4257-997b-9524ad774b1a None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:10.831765 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de45f392-74b3-4257-997b-9524ad774b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:10.849368 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-de45f392-74b3-4257-997b-9524ad774b1a None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:10.850545 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-de45f392-74b3-4257-997b-9524ad774b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:10.929033 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 557/1132] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:11:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:10.963155 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb4780-4c0a-4eb5-897f-78999825c018 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:11.067018 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbb4780-4c0a-4eb5-897f-78999825c018 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:11.102463 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a65a00-4b3a-4efb-9a24-80d60fb7e700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:11.103188 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a65a00-4b3a-4efb-9a24-80d60fb7e700 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:11.103468 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a65a00-4b3a-4efb-9a24-80d60fb7e700 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:11.103735 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a65a00-4b3a-4efb-9a24-80d60fb7e700 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:11.105060 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a65a00-4b3a-4efb-9a24-80d60fb7e700 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:11.106513 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a65a00-4b3a-4efb-9a24-80d60fb7e700 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:11.108127 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 576/1133] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:11:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:11.114892 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45f392-74b3-4257-997b-9524ad774b1a None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:11.226926 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de45f392-74b3-4257-997b-9524ad774b1a None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:11.228518 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:11.263908 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c749c2a8-b412-433b-a4a7-6489130decfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:11.264559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749c2a8-b412-433b-a4a7-6489130decfa None admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:11.264839 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749c2a8-b412-433b-a4a7-6489130decfa None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:11.265141 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749c2a8-b412-433b-a4a7-6489130decfa None admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:11.265848 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c749c2a8-b412-433b-a4a7-6489130decfa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:11.267284 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c749c2a8-b412-433b-a4a7-6489130decfa None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:11.299203 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 558/1134] 10.222.0.50 () {60 vars in 1141 bytes} [Fri May 27 03:11:11 2022] POST /identity/v3/projects => generated 311 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:11:13.689734 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd06ed-dbeb-447b-bf20-7610f3edf3e2 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:13.690610 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd06ed-dbeb-447b-bf20-7610f3edf3e2 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:13.690610 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd06ed-dbeb-447b-bf20-7610f3edf3e2 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:13.692906 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 577/1135] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:11:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:13.699069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbfaa8a-36a0-47c9-aafd-29fe38147f89 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:13.699571 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbfaa8a-36a0-47c9-aafd-29fe38147f89 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:13.699571 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbfaa8a-36a0-47c9-aafd-29fe38147f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:13.719850 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-afbfaa8a-36a0-47c9-aafd-29fe38147f89 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:13.721358 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-afbfaa8a-36a0-47c9-aafd-29fe38147f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:13.796595 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 559/1136] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:11:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:13.822749 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd06ed-dbeb-447b-bf20-7610f3edf3e2 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:13.880319 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:13.947170 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd06ed-dbeb-447b-bf20-7610f3edf3e2 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:13.986105 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953ed45-2ef3-406e-adee-e9f3e706bcb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:13.986917 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953ed45-2ef3-406e-adee-e9f3e706bcb1 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:13.987172 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953ed45-2ef3-406e-adee-e9f3e706bcb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:13.987441 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953ed45-2ef3-406e-adee-e9f3e706bcb1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:13.988742 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1953ed45-2ef3-406e-adee-e9f3e706bcb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:13.990240 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1953ed45-2ef3-406e-adee-e9f3e706bcb1 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:13.991886 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 578/1137] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:11:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:14.001208 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbfaa8a-36a0-47c9-aafd-29fe38147f89 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:14.087302 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:14.104973 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbfaa8a-36a0-47c9-aafd-29fe38147f89 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:14.148522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e56b6b-dd34-494d-afe0-b0493dd28b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:14.149306 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e56b6b-dd34-494d-afe0-b0493dd28b57 None admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:14.149646 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e56b6b-dd34-494d-afe0-b0493dd28b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:14.150115 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e56b6b-dd34-494d-afe0-b0493dd28b57 None admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:14.151271 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e56b6b-dd34-494d-afe0-b0493dd28b57 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:14.153653 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e56b6b-dd34-494d-afe0-b0493dd28b57 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:14.178257 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 560/1138] 10.222.0.50 () {60 vars in 1132 bytes} [Fri May 27 03:11:13 2022] POST /identity/v3/users => generated 286 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:11:16.398704 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35ebdc-ccf9-4a77-86c2-218f829ac897 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:16.399587 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35ebdc-ccf9-4a77-86c2-218f829ac897 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:16.399587 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf35ebdc-ccf9-4a77-86c2-218f829ac897 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:16.401846 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 579/1139] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:11:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:16.408849 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f4f4b-dace-413c-8cf2-1212834e3795 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:16.409718 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f4f4b-dace-413c-8cf2-1212834e3795 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:16.410003 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f4f4b-dace-413c-8cf2-1212834e3795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:16.428345 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-f82f4f4b-dace-413c-8cf2-1212834e3795 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:16.429573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-f82f4f4b-dace-413c-8cf2-1212834e3795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:16.499539 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 561/1140] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:11:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:16.523528 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35ebdc-ccf9-4a77-86c2-218f829ac897 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:16.643164 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf35ebdc-ccf9-4a77-86c2-218f829ac897 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:16.657124 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:16.688562 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d17aceb-8006-4869-8d55-df8df94fe776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:16.689361 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17aceb-8006-4869-8d55-df8df94fe776 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:16.689668 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17aceb-8006-4869-8d55-df8df94fe776 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:16.689954 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17aceb-8006-4869-8d55-df8df94fe776 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:16.700792 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d17aceb-8006-4869-8d55-df8df94fe776 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:11:16.708368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-0d17aceb-8006-4869-8d55-df8df94fe776 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:11:16.708368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:16.708368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:16.708368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:16.708368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:16.708368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:16.708368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:16.708368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:16.708368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:16.708368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:16.708368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:16.708368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:16.708368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:11:16.708368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:11:16.708368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:11:16.708368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:11:16.709429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:11:16.709429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:11:16.709429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:11:16.709429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:11:16.709429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:16.709429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:16.709429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:16.709429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:16.709429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:16.709429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:16.709429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:16.709429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:11:16.709429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:16.709429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:16.709429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:16.709429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:16.710402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:16.710402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:16.710402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:16.710402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:16.710402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:11:16.710402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:11:16.710402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:11:16.710402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:11:16.710402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:11:16.710402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:11:16.710402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:11:16.710402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:11:16.710402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 580/1141] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 03:11:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:16.716054 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f4f4b-dace-413c-8cf2-1212834e3795 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:16.805325 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:16.839793 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f4f4b-dace-413c-8cf2-1212834e3795 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:16.874437 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad4246-1627-4390-a9f5-79279db102fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:16.875093 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad4246-1627-4390-a9f5-79279db102fd None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:16.875367 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad4246-1627-4390-a9f5-79279db102fd None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:16.875636 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad4246-1627-4390-a9f5-79279db102fd None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:16.876373 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ad4246-1627-4390-a9f5-79279db102fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:16.876676 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ad4246-1627-4390-a9f5-79279db102fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:16.878277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ad4246-1627-4390-a9f5-79279db102fd None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:16.886481 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 562/1142] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:11:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:16.894809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d17aceb-8006-4869-8d55-df8df94fe776 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:17.007524 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d17aceb-8006-4869-8d55-df8df94fe776 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:17.045443 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a77313b-191a-44b0-935a-d0ebe11d0b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:17.046309 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a77313b-191a-44b0-935a-d0ebe11d0b90 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:17.046584 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a77313b-191a-44b0-935a-d0ebe11d0b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:17.046856 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a77313b-191a-44b0-935a-d0ebe11d0b90 None admin] PATH_INFO: `/v3/users/9b1ff74100ba4d37ae007a8f5060b55e` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:17.060981 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a77313b-191a-44b0-935a-d0ebe11d0b90 None admin] RBAC: Authorizing `identity:get_user(user_id=9b1ff74100ba4d37ae007a8f5060b55e)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:17.062457 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a77313b-191a-44b0-935a-d0ebe11d0b90 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:17.064098 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 581/1143] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:11:16 2022] GET /identity/v3/users/9b1ff74100ba4d37ae007a8f5060b55e => generated 286 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:17.071001 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad4246-1627-4390-a9f5-79279db102fd None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:17.190551 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad4246-1627-4390-a9f5-79279db102fd None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:17.220970 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:17.226858 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef6fa6-afc9-425b-a898-73fad2d5f2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:17.227501 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef6fa6-afc9-425b-a898-73fad2d5f2ae None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:17.227778 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef6fa6-afc9-425b-a898-73fad2d5f2ae None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:17.228049 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef6fa6-afc9-425b-a898-73fad2d5f2ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:17.296632 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ef6fa6-afc9-425b-a898-73fad2d5f2ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:17.298212 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ef6fa6-afc9-425b-a898-73fad2d5f2ae None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:17.406765 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 563/1144] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:11:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:11:17.416683 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a77313b-191a-44b0-935a-d0ebe11d0b90 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:17.455253 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:17.552226 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a77313b-191a-44b0-935a-d0ebe11d0b90 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:17.587774 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea38024-bcc7-43bb-9450-dfee999b4cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:17.588631 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea38024-bcc7-43bb-9450-dfee999b4cbf None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:17.588909 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea38024-bcc7-43bb-9450-dfee999b4cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:17.589218 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea38024-bcc7-43bb-9450-dfee999b4cbf None admin] PATH_INFO: `/v3/projects/ae814aa91a064732ac56b24a31d1ed57` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:17.590426 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea38024-bcc7-43bb-9450-dfee999b4cbf None admin] RBAC: Authorizing `identity:get_project(project_id=ae814aa91a064732ac56b24a31d1ed57)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:17.591873 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea38024-bcc7-43bb-9450-dfee999b4cbf None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:17.593605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 582/1145] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:11:17 2022] GET /identity/v3/projects/ae814aa91a064732ac56b24a31d1ed57 => generated 311 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:17.604526 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef6fa6-afc9-425b-a898-73fad2d5f2ae None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:17.640516 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:17.714031 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef6fa6-afc9-425b-a898-73fad2d5f2ae None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:17.749116 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a09773e-73e4-4165-9113-e535b95de76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:17.749764 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a09773e-73e4-4165-9113-e535b95de76f None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:17.750045 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a09773e-73e4-4165-9113-e535b95de76f None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:17.750322 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a09773e-73e4-4165-9113-e535b95de76f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:17.751150 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a09773e-73e4-4165-9113-e535b95de76f None admin] RBAC: Adding query filter params (role.id=b4c524dd7bb44e94bd3b42d86e1cdf42, scope.project.id=ae814aa91a064732ac56b24a31d1ed57, user.id=9b1ff74100ba4d37ae007a8f5060b55e) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:17.751466 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a09773e-73e4-4165-9113-e535b95de76f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:17.753094 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a09773e-73e4-4165-9113-e535b95de76f None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:17.758525 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 564/1146] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:11:17 2022] GET /identity/v3/role_assignments?user.id=9b1ff74100ba4d37ae007a8f5060b55e&scope.project.id=ae814aa91a064732ac56b24a31d1ed57&role.id=b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:19.916166 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337e609-6255-41bb-860a-a7d9310c010d None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:19.916784 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337e609-6255-41bb-860a-a7d9310c010d None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:19.916784 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337e609-6255-41bb-860a-a7d9310c010d None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:19.918127 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 583/1147] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:11:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:19.923667 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a84a6c-41ab-48d8-873d-f1ff1c81af81 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:19.924124 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a84a6c-41ab-48d8-873d-f1ff1c81af81 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:19.924223 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a84a6c-41ab-48d8-873d-f1ff1c81af81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:19.942983 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-f5a84a6c-41ab-48d8-873d-f1ff1c81af81 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:19.944179 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-f5a84a6c-41ab-48d8-873d-f1ff1c81af81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:20.015325 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 565/1148] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:11:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:20.049312 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337e609-6255-41bb-860a-a7d9310c010d None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:20.160082 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:20.165289 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337e609-6255-41bb-860a-a7d9310c010d None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:20.201170 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e2e51f-1fe4-4f68-925c-6ad26835199f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:20.201955 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2e51f-1fe4-4f68-925c-6ad26835199f None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:20.202234 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2e51f-1fe4-4f68-925c-6ad26835199f None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:20.202511 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e2e51f-1fe4-4f68-925c-6ad26835199f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:20.214793 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e2e51f-1fe4-4f68-925c-6ad26835199f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:11:20.222631 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-94e2e51f-1fe4-4f68-925c-6ad26835199f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:11:20.222631 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:20.222631 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:20.222631 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:20.222631 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:20.222631 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:20.222631 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:20.222631 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:20.222631 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:20.222631 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:20.222631 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:20.222631 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:20.222631 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:11:20.222631 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:11:20.222631 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:11:20.222631 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:11:20.223832 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:11:20.223832 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:11:20.223832 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:11:20.223832 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:11:20.223832 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:20.223832 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:20.223832 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:20.223832 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:20.223832 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:20.223832 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:20.223832 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:20.223832 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:11:20.223832 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:20.223832 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:20.223832 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:20.223832 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:20.224737 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:20.224737 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:20.224737 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:20.224737 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:20.224737 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:11:20.224737 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:11:20.224737 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:11:20.224737 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:11:20.224737 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:11:20.224737 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:11:20.224737 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:11:20.224737 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:11:20.224737 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 584/1149] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 03:11:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:20.231753 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a84a6c-41ab-48d8-873d-f1ff1c81af81 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:20.259212 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:20.335289 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a84a6c-41ab-48d8-873d-f1ff1c81af81 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:20.371353 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71778390-4c83-4ddc-8a5b-17f6ef502c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:20.372028 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71778390-4c83-4ddc-8a5b-17f6ef502c31 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:20.372310 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71778390-4c83-4ddc-8a5b-17f6ef502c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:20.372584 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71778390-4c83-4ddc-8a5b-17f6ef502c31 None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:20.373927 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71778390-4c83-4ddc-8a5b-17f6ef502c31 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:20.374461 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71778390-4c83-4ddc-8a5b-17f6ef502c31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:20.377001 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71778390-4c83-4ddc-8a5b-17f6ef502c31 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:20.389406 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 566/1150] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:11:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:20.396516 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e2e51f-1fe4-4f68-925c-6ad26835199f None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:20.523963 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e2e51f-1fe4-4f68-925c-6ad26835199f None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:20.560278 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e720f6-3cec-4000-ad2a-8a9fca71918f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:20.561175 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e720f6-3cec-4000-ad2a-8a9fca71918f None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:20.561449 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e720f6-3cec-4000-ad2a-8a9fca71918f None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:20.561714 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e720f6-3cec-4000-ad2a-8a9fca71918f None admin] PATH_INFO: `/v3/users/9b1ff74100ba4d37ae007a8f5060b55e` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:20.562939 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e720f6-3cec-4000-ad2a-8a9fca71918f None admin] RBAC: Authorizing `identity:get_user(user_id=9b1ff74100ba4d37ae007a8f5060b55e)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:20.564430 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e720f6-3cec-4000-ad2a-8a9fca71918f None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:20.566072 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 585/1151] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:11:20 2022] GET /identity/v3/users/9b1ff74100ba4d37ae007a8f5060b55e => generated 286 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:20.571969 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71778390-4c83-4ddc-8a5b-17f6ef502c31 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:20.666402 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:20.679634 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71778390-4c83-4ddc-8a5b-17f6ef502c31 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:20.714514 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a547e190-6e90-4c00-986f-7faf77fafcd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:20.715374 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a547e190-6e90-4c00-986f-7faf77fafcd2 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:20.715652 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a547e190-6e90-4c00-986f-7faf77fafcd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:20.715925 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a547e190-6e90-4c00-986f-7faf77fafcd2 None admin] PATH_INFO: `/v3/projects/ae814aa91a064732ac56b24a31d1ed57` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:20.717254 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a547e190-6e90-4c00-986f-7faf77fafcd2 None admin] RBAC: Authorizing `identity:get_project(project_id=ae814aa91a064732ac56b24a31d1ed57)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:20.718685 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a547e190-6e90-4c00-986f-7faf77fafcd2 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:20.720398 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 567/1152] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:11:20 2022] GET /identity/v3/projects/ae814aa91a064732ac56b24a31d1ed57 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:20.727215 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e720f6-3cec-4000-ad2a-8a9fca71918f None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:20.767905 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:20.853568 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e720f6-3cec-4000-ad2a-8a9fca71918f None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:20.897968 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e711db9-8c9c-4018-a5c0-f46cd1ac631d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:20.899338 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e711db9-8c9c-4018-a5c0-f46cd1ac631d None admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:20.899720 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e711db9-8c9c-4018-a5c0-f46cd1ac631d None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:20.900119 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e711db9-8c9c-4018-a5c0-f46cd1ac631d None admin] PATH_INFO: `/v3/projects/ae814aa91a064732ac56b24a31d1ed57/users/9b1ff74100ba4d37ae007a8f5060b55e/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:20.903676 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e711db9-8c9c-4018-a5c0-f46cd1ac631d None admin] RBAC: Authorizing `identity:create_grant(project_id=ae814aa91a064732ac56b24a31d1ed57, user_id=9b1ff74100ba4d37ae007a8f5060b55e, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:20.906044 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e711db9-8c9c-4018-a5c0-f46cd1ac631d None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:20.927579 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 586/1153] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 03:11:20 2022] PUT /identity/v3/projects/ae814aa91a064732ac56b24a31d1ed57/users/9b1ff74100ba4d37ae007a8f5060b55e/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:11:23.141525 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76339e82-5191-4e6a-98a1-97e650e414f3 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:23.142422 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76339e82-5191-4e6a-98a1-97e650e414f3 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:23.142422 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76339e82-5191-4e6a-98a1-97e650e414f3 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:23.144700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 568/1154] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:11:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:23.151143 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e55d21-5f3b-4cf4-ac28-b38b7fd32a4f None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:23.151925 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e55d21-5f3b-4cf4-ac28-b38b7fd32a4f None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:23.152090 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e55d21-5f3b-4cf4-ac28-b38b7fd32a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:23.173521 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-a4e55d21-5f3b-4cf4-ac28-b38b7fd32a4f None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:23.174794 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-a4e55d21-5f3b-4cf4-ac28-b38b7fd32a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:23.247313 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 587/1155] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:11:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:23.272324 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76339e82-5191-4e6a-98a1-97e650e414f3 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:23.391280 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76339e82-5191-4e6a-98a1-97e650e414f3 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:23.426087 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bac7f8-6fc2-4247-b0e2-97273fe7d0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:23.426884 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bac7f8-6fc2-4247-b0e2-97273fe7d0ea None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:23.427163 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bac7f8-6fc2-4247-b0e2-97273fe7d0ea None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:23.427434 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bac7f8-6fc2-4247-b0e2-97273fe7d0ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:23.439371 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bac7f8-6fc2-4247-b0e2-97273fe7d0ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:11:23.447121 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-45bac7f8-6fc2-4247-b0e2-97273fe7d0ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:11:23.447121 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:23.447121 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:23.447121 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:23.447121 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:23.447121 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:23.447121 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:23.447121 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:23.447121 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:23.447121 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:23.447121 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:23.447121 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:23.447121 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:11:23.447121 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:11:23.447121 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:11:23.447121 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:11:23.448337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:11:23.448337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:11:23.448337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:11:23.448337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:11:23.448337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:23.448337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:23.448337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:23.448337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:23.448337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:23.448337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:23.448337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:23.448337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:11:23.448337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:23.448337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:23.448337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:23.448337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:23.449388 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:23.449388 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:23.449388 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:23.449388 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:23.449388 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:11:23.449388 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:11:23.449388 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:11:23.449388 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:11:23.449388 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:11:23.449388 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:11:23.449388 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:11:23.449388 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:11:23.449388 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 569/1156] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 03:11:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:23.457357 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e55d21-5f3b-4cf4-ac28-b38b7fd32a4f None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:23.480394 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:23.572157 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e55d21-5f3b-4cf4-ac28-b38b7fd32a4f None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:23.609504 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0627f45-2786-4149-aa56-d074c21da31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:23.610169 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0627f45-2786-4149-aa56-d074c21da31d None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:23.610443 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0627f45-2786-4149-aa56-d074c21da31d None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:23.610717 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0627f45-2786-4149-aa56-d074c21da31d None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:23.611458 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0627f45-2786-4149-aa56-d074c21da31d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:23.611764 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0627f45-2786-4149-aa56-d074c21da31d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:23.613462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0627f45-2786-4149-aa56-d074c21da31d None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:23.621663 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 588/1157] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:11:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:23.627921 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bac7f8-6fc2-4247-b0e2-97273fe7d0ea None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:23.648729 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:23.744674 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bac7f8-6fc2-4247-b0e2-97273fe7d0ea None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:23.790643 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee649bf-8a56-48e2-9c1a-6098b5d0576b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:23.791533 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee649bf-8a56-48e2-9c1a-6098b5d0576b None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:23.791807 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee649bf-8a56-48e2-9c1a-6098b5d0576b None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:23.792075 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee649bf-8a56-48e2-9c1a-6098b5d0576b None admin] PATH_INFO: `/v3/users/9b1ff74100ba4d37ae007a8f5060b55e` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:23.793425 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee649bf-8a56-48e2-9c1a-6098b5d0576b None admin] RBAC: Authorizing `identity:get_user(user_id=9b1ff74100ba4d37ae007a8f5060b55e)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:23.794889 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee649bf-8a56-48e2-9c1a-6098b5d0576b None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:23.796529 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 570/1158] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:11:23 2022] GET /identity/v3/users/9b1ff74100ba4d37ae007a8f5060b55e => generated 286 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:23.803343 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0627f45-2786-4149-aa56-d074c21da31d None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:23.896188 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:23.929345 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0627f45-2786-4149-aa56-d074c21da31d None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:23.965573 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b703d9a-98b1-4b8e-ba89-5f9205f11166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:23.966226 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b703d9a-98b1-4b8e-ba89-5f9205f11166 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:23.966506 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b703d9a-98b1-4b8e-ba89-5f9205f11166 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:23.966778 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b703d9a-98b1-4b8e-ba89-5f9205f11166 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:24.037662 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b703d9a-98b1-4b8e-ba89-5f9205f11166 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:24.039052 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b703d9a-98b1-4b8e-ba89-5f9205f11166 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:24.117951 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:24.146084 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 589/1159] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:11:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:11:24.155680 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee649bf-8a56-48e2-9c1a-6098b5d0576b None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:24.255596 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:24.273801 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee649bf-8a56-48e2-9c1a-6098b5d0576b None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:24.315972 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c579e-87d4-4d93-a6ff-112330d47c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:24.316839 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c579e-87d4-4d93-a6ff-112330d47c6f None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:24.317154 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c579e-87d4-4d93-a6ff-112330d47c6f None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:24.317425 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c579e-87d4-4d93-a6ff-112330d47c6f None admin] PATH_INFO: `/v3/projects/ae814aa91a064732ac56b24a31d1ed57` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:24.318601 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8c579e-87d4-4d93-a6ff-112330d47c6f None admin] RBAC: Authorizing `identity:get_project(project_id=ae814aa91a064732ac56b24a31d1ed57)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:24.320042 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8c579e-87d4-4d93-a6ff-112330d47c6f None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:24.321772 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 571/1160] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:11:24 2022] GET /identity/v3/projects/ae814aa91a064732ac56b24a31d1ed57 => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:24.331740 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b703d9a-98b1-4b8e-ba89-5f9205f11166 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:24.437521 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b703d9a-98b1-4b8e-ba89-5f9205f11166 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:24.483028 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e3cac-4835-4518-b5cd-40192008077c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:24.484048 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e3cac-4835-4518-b5cd-40192008077c None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:24.484482 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e3cac-4835-4518-b5cd-40192008077c None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:24.484895 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e3cac-4835-4518-b5cd-40192008077c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:24.486263 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60e3cac-4835-4518-b5cd-40192008077c None admin] RBAC: Adding query filter params (role.id=b4c524dd7bb44e94bd3b42d86e1cdf42, scope.project.id=ae814aa91a064732ac56b24a31d1ed57, user.id=9b1ff74100ba4d37ae007a8f5060b55e) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:24.486752 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60e3cac-4835-4518-b5cd-40192008077c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:24.489318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60e3cac-4835-4518-b5cd-40192008077c None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:24.498932 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 590/1161] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:11:24 2022] GET /identity/v3/role_assignments?user.id=9b1ff74100ba4d37ae007a8f5060b55e&scope.project.id=ae814aa91a064732ac56b24a31d1ed57&role.id=b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:26.778997 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7528c2c-3e2b-4287-876e-25c41ad5e694 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:26.779891 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7528c2c-3e2b-4287-876e-25c41ad5e694 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:26.779891 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7528c2c-3e2b-4287-876e-25c41ad5e694 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:26.782100 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 572/1162] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:11:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:26.789847 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da474383-1a72-446f-940f-de92f6383d31 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:26.790277 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da474383-1a72-446f-940f-de92f6383d31 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:26.790619 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da474383-1a72-446f-940f-de92f6383d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:26.818403 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-da474383-1a72-446f-940f-de92f6383d31 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:26.819586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-da474383-1a72-446f-940f-de92f6383d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:26.857266 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:26.891046 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 591/1163] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:11:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:26.914044 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7528c2c-3e2b-4287-876e-25c41ad5e694 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:27.024390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7528c2c-3e2b-4287-876e-25c41ad5e694 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:27.063960 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9039a749-f581-4fc8-bef1-69b826c54ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:27.064520 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039a749-f581-4fc8-bef1-69b826c54ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:27.064797 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039a749-f581-4fc8-bef1-69b826c54ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:27.065188 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039a749-f581-4fc8-bef1-69b826c54ec6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:27.088760 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:27.136338 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9039a749-f581-4fc8-bef1-69b826c54ec6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:27.137805 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9039a749-f581-4fc8-bef1-69b826c54ec6 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:27.237586 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 573/1164] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:11:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:11:27.244913 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da474383-1a72-446f-940f-de92f6383d31 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:27.355027 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da474383-1a72-446f-940f-de92f6383d31 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:27.390788 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4800cae-dbe4-4419-ae4a-3b9b1b11a2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:27.391585 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4800cae-dbe4-4419-ae4a-3b9b1b11a2ec None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:27.391868 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4800cae-dbe4-4419-ae4a-3b9b1b11a2ec None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:27.392139 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4800cae-dbe4-4419-ae4a-3b9b1b11a2ec None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:27.399013 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4800cae-dbe4-4419-ae4a-3b9b1b11a2ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:27.400641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4800cae-dbe4-4419-ae4a-3b9b1b11a2ec None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:27.415520 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-f4800cae-dbe4-4419-ae4a-3b9b1b11a2ec None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 27 03:11:27.415520 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:27.415520 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 27 03:11:27.415520 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 27 03:11:27.415520 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:11:27.415520 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:11:27.415520 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:11:27.415520 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:11:27.415520 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 27 03:11:27.415520 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:11:27.415520 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:11:27.415520 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:11:27.415520 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:27.415520 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:27.415520 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:27.415520 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:27.415520 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:27.416711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:27.416711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:27.416711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:27.416711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:27.416711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:27.416711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:27.416711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:11:27.416711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:11:27.416711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:11:27.416711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:11:27.416711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:27.416711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:27.416711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:27.416711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:27.416711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:27.416711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:27.417649 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:27.417649 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:11:27.417649 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:27.417649 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:27.417649 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:27.417649 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:27.417649 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:27.417649 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:27.417649 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:27.417649 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:27.417649 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 27 03:11:27.417649 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:11:27.417649 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 27 03:11:27.417649 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:11:27.417649 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 592/1165] 10.222.0.50 () {56 vars in 1117 bytes} [Fri May 27 03:11:27 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:27.424909 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9039a749-f581-4fc8-bef1-69b826c54ec6 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:27.483762 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:27.539143 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9039a749-f581-4fc8-bef1-69b826c54ec6 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:27.574033 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf214b0-57c1-46ab-82d1-2538286e33ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:27.574679 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf214b0-57c1-46ab-82d1-2538286e33ea None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:27.574953 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf214b0-57c1-46ab-82d1-2538286e33ea None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:27.575223 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf214b0-57c1-46ab-82d1-2538286e33ea None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:27.575942 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf214b0-57c1-46ab-82d1-2538286e33ea None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:27.576238 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf214b0-57c1-46ab-82d1-2538286e33ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:27.577630 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf214b0-57c1-46ab-82d1-2538286e33ea None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:27.585049 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 574/1166] 10.222.0.50 () {56 vars in 1115 bytes} [Fri May 27 03:11:27 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:11:29.752791 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb9afe7-1b9e-47a0-9966-75020171bd31 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:29.753786 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb9afe7-1b9e-47a0-9966-75020171bd31 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:29.753786 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb9afe7-1b9e-47a0-9966-75020171bd31 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:29.755984 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 593/1167] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:11:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:29.762612 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddd9ed1-4072-48db-a0a1-1895e8a593e9 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:29.763372 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddd9ed1-4072-48db-a0a1-1895e8a593e9 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:29.763540 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddd9ed1-4072-48db-a0a1-1895e8a593e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:29.789713 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-7ddd9ed1-4072-48db-a0a1-1895e8a593e9 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:29.790902 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-7ddd9ed1-4072-48db-a0a1-1895e8a593e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:29.866044 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 575/1168] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:11:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:29.892987 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb9afe7-1b9e-47a0-9966-75020171bd31 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:29.929675 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:30.010552 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb9afe7-1b9e-47a0-9966-75020171bd31 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:30.048748 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474492b-56d2-49df-9c16-9c4804590b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:30.049983 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474492b-56d2-49df-9c16-9c4804590b7e None admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:30.050254 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474492b-56d2-49df-9c16-9c4804590b7e None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:30.050527 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474492b-56d2-49df-9c16-9c4804590b7e None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:30.051255 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f474492b-56d2-49df-9c16-9c4804590b7e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:30.052632 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f474492b-56d2-49df-9c16-9c4804590b7e None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:30.071401 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 594/1169] 10.222.0.50 () {60 vars in 1138 bytes} [Fri May 27 03:11:29 2022] POST /identity/v3/domains => generated 267 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:11:32.293338 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f092a8f-6aec-45b9-b6f4-ffdf6f3a6894 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:32.293874 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f092a8f-6aec-45b9-b6f4-ffdf6f3a6894 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:32.293874 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f092a8f-6aec-45b9-b6f4-ffdf6f3a6894 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:32.295320 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 576/1170] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:11:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:32.302257 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d1a44-d27c-4c8b-8347-bea5bf19987d None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:32.302934 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d1a44-d27c-4c8b-8347-bea5bf19987d None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:32.303024 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9d1a44-d27c-4c8b-8347-bea5bf19987d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:32.324998 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-5f9d1a44-d27c-4c8b-8347-bea5bf19987d None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:32.326214 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-5f9d1a44-d27c-4c8b-8347-bea5bf19987d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:32.395887 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 595/1171] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:11:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:32.420442 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f092a8f-6aec-45b9-b6f4-ffdf6f3a6894 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:32.476466 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:32.533857 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f092a8f-6aec-45b9-b6f4-ffdf6f3a6894 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:32.568352 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d09012-4f09-46c9-ba46-2e1312e7b455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:32.569169 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d09012-4f09-46c9-ba46-2e1312e7b455 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:32.569447 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d09012-4f09-46c9-ba46-2e1312e7b455 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:32.569724 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d09012-4f09-46c9-ba46-2e1312e7b455 None admin] PATH_INFO: `/v3/domains/7c1474e877f54286bba9c1d681d15664` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:32.580760 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d09012-4f09-46c9-ba46-2e1312e7b455 None admin] RBAC: Authorizing `identity:get_domain(domain_id=7c1474e877f54286bba9c1d681d15664)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:32.582288 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d09012-4f09-46c9-ba46-2e1312e7b455 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:32.583945 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 577/1172] 10.222.0.50 () {56 vars in 1183 bytes} [Fri May 27 03:11:32 2022] GET /identity/v3/domains/7c1474e877f54286bba9c1d681d15664 => generated 267 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:32.590582 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d1a44-d27c-4c8b-8347-bea5bf19987d None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:32.623327 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:32.706099 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9d1a44-d27c-4c8b-8347-bea5bf19987d None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:32.747043 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94b472-466d-4a62-841d-e39776104206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:32.747708 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94b472-466d-4a62-841d-e39776104206 None admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:32.747986 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94b472-466d-4a62-841d-e39776104206 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:32.748259 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94b472-466d-4a62-841d-e39776104206 None admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:32.748976 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f94b472-466d-4a62-841d-e39776104206 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:32.750430 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f94b472-466d-4a62-841d-e39776104206 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:32.781159 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 596/1173] 10.222.0.50 () {60 vars in 1141 bytes} [Fri May 27 03:11:32 2022] POST /identity/v3/projects => generated 361 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:11:35.061210 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8edd49-e308-480e-864f-a01ecd3ed63a None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:35.061777 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8edd49-e308-480e-864f-a01ecd3ed63a None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:35.061777 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8edd49-e308-480e-864f-a01ecd3ed63a None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:35.063155 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 578/1174] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:11:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:35.070304 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1876e-2c7b-4e99-88df-f52d6096f961 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:35.071020 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1876e-2c7b-4e99-88df-f52d6096f961 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:35.071197 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1876e-2c7b-4e99-88df-f52d6096f961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:35.098149 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-b5f1876e-2c7b-4e99-88df-f52d6096f961 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:35.099339 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-b5f1876e-2c7b-4e99-88df-f52d6096f961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:35.169550 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 597/1175] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:11:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:35.194940 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8edd49-e308-480e-864f-a01ecd3ed63a None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:35.309305 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8edd49-e308-480e-864f-a01ecd3ed63a None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:35.317080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:35.353387 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2763ab54-a67c-458e-8eec-4546ed1e741a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:35.354254 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763ab54-a67c-458e-8eec-4546ed1e741a None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:35.354538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763ab54-a67c-458e-8eec-4546ed1e741a None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:35.354810 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2763ab54-a67c-458e-8eec-4546ed1e741a None admin] PATH_INFO: `/v3/domains/7c1474e877f54286bba9c1d681d15664` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:35.356065 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2763ab54-a67c-458e-8eec-4546ed1e741a None admin] RBAC: Authorizing `identity:get_domain(domain_id=7c1474e877f54286bba9c1d681d15664)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:35.357547 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2763ab54-a67c-458e-8eec-4546ed1e741a None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:35.359175 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 579/1176] 10.222.0.50 () {56 vars in 1183 bytes} [Fri May 27 03:11:35 2022] GET /identity/v3/domains/7c1474e877f54286bba9c1d681d15664 => generated 267 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:35.366030 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1876e-2c7b-4e99-88df-f52d6096f961 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:35.369387 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:35.474100 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1876e-2c7b-4e99-88df-f52d6096f961 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:35.509331 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef74f15-b7b1-4036-84a2-fcc935b2fa0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:35.510040 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef74f15-b7b1-4036-84a2-fcc935b2fa0f None admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:35.510317 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef74f15-b7b1-4036-84a2-fcc935b2fa0f None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:35.510587 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef74f15-b7b1-4036-84a2-fcc935b2fa0f None admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:35.511288 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef74f15-b7b1-4036-84a2-fcc935b2fa0f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:35.512762 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef74f15-b7b1-4036-84a2-fcc935b2fa0f None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:35.536142 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 598/1177] 10.222.0.50 () {60 vars in 1132 bytes} [Fri May 27 03:11:35 2022] POST /identity/v3/users => generated 311 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:11:37.752148 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c280bd8e-7c40-4c6c-801a-5fa4d8725273 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:37.752735 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c280bd8e-7c40-4c6c-801a-5fa4d8725273 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:37.752735 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c280bd8e-7c40-4c6c-801a-5fa4d8725273 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:37.754123 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 580/1178] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:11:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:37.760597 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2ec6c-5260-4853-aeee-9e784afd9cbd None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:37.761351 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2ec6c-5260-4853-aeee-9e784afd9cbd None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:37.761525 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2ec6c-5260-4853-aeee-9e784afd9cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:37.783642 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-27a2ec6c-5260-4853-aeee-9e784afd9cbd None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:37.784832 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-27a2ec6c-5260-4853-aeee-9e784afd9cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:37.847855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:37.861825 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 599/1179] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:11:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:37.885553 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c280bd8e-7c40-4c6c-801a-5fa4d8725273 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:38.009667 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c280bd8e-7c40-4c6c-801a-5fa4d8725273 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:38.045417 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859904af-b92c-45ee-be1c-b707987c0e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:38.046208 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859904af-b92c-45ee-be1c-b707987c0e51 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:38.046493 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859904af-b92c-45ee-be1c-b707987c0e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:38.046769 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859904af-b92c-45ee-be1c-b707987c0e51 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:38.058525 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859904af-b92c-45ee-be1c-b707987c0e51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:11:38.066288 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-859904af-b92c-45ee-be1c-b707987c0e51 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:11:38.066288 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:38.066288 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:38.066288 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:38.066288 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:38.066288 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:38.066288 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:38.066288 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:38.066288 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:38.066288 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:38.066288 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:38.066288 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:38.066288 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:11:38.066288 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:11:38.066288 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:11:38.066288 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:11:38.067355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:11:38.067355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:11:38.067355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:11:38.067355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:11:38.067355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:38.067355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:38.067355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:38.067355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:38.067355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:38.067355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:38.067355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:38.067355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:11:38.067355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:38.067355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:38.067355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:38.067355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:38.068435 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:38.068435 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:38.068435 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:38.068435 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:38.068435 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:11:38.068435 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:11:38.068435 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:11:38.068435 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:11:38.068435 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:11:38.068435 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:11:38.068435 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:11:38.068435 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:11:38.068435 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 581/1180] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 03:11:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:38.074546 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a2ec6c-5260-4853-aeee-9e784afd9cbd None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:38.199177 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a2ec6c-5260-4853-aeee-9e784afd9cbd None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:38.242460 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9df478-fc38-4136-adea-b1b050565977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:38.243127 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9df478-fc38-4136-adea-b1b050565977 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:38.243403 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9df478-fc38-4136-adea-b1b050565977 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:38.243672 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9df478-fc38-4136-adea-b1b050565977 None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:38.244414 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9df478-fc38-4136-adea-b1b050565977 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:38.244719 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9df478-fc38-4136-adea-b1b050565977 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:38.246140 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9df478-fc38-4136-adea-b1b050565977 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:38.254634 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 600/1181] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:11:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:11:38.260886 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859904af-b92c-45ee-be1c-b707987c0e51 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:38.298261 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:38.368780 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859904af-b92c-45ee-be1c-b707987c0e51 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:38.403856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93952db9-b3e1-424b-9b8c-ba1768480bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:38.404729 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93952db9-b3e1-424b-9b8c-ba1768480bfe None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:38.404989 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93952db9-b3e1-424b-9b8c-ba1768480bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:38.405294 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93952db9-b3e1-424b-9b8c-ba1768480bfe None admin] PATH_INFO: `/v3/users/5fb3166f1b4e42bcbd121a394f32ded5` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:38.419527 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93952db9-b3e1-424b-9b8c-ba1768480bfe None admin] RBAC: Authorizing `identity:get_user(user_id=5fb3166f1b4e42bcbd121a394f32ded5)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:38.420985 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93952db9-b3e1-424b-9b8c-ba1768480bfe None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:38.422634 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 582/1182] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:11:38 2022] GET /identity/v3/users/5fb3166f1b4e42bcbd121a394f32ded5 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:38.429491 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9df478-fc38-4136-adea-b1b050565977 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:38.486266 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:38.540308 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9df478-fc38-4136-adea-b1b050565977 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:38.574908 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3596827-eb3f-4071-9dc9-c0186a1b8015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:38.575541 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3596827-eb3f-4071-9dc9-c0186a1b8015 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:38.575813 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3596827-eb3f-4071-9dc9-c0186a1b8015 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:38.576080 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3596827-eb3f-4071-9dc9-c0186a1b8015 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:38.652565 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3596827-eb3f-4071-9dc9-c0186a1b8015 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:38.654071 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3596827-eb3f-4071-9dc9-c0186a1b8015 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:38.714538 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:38.765005 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 601/1183] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:11:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:11:38.775167 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93952db9-b3e1-424b-9b8c-ba1768480bfe None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:38.878811 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:38.894686 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93952db9-b3e1-424b-9b8c-ba1768480bfe None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:38.937072 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c669e4ea-ad13-4324-b9c3-0b7dcb0edc5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:38.937928 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669e4ea-ad13-4324-b9c3-0b7dcb0edc5e None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:38.938203 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669e4ea-ad13-4324-b9c3-0b7dcb0edc5e None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:38.938476 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669e4ea-ad13-4324-b9c3-0b7dcb0edc5e None admin] PATH_INFO: `/v3/projects/bece55739d5e4f14bbb05b57f5e7a495` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:38.939758 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c669e4ea-ad13-4324-b9c3-0b7dcb0edc5e None admin] RBAC: Authorizing `identity:get_project(project_id=bece55739d5e4f14bbb05b57f5e7a495)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:38.941242 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c669e4ea-ad13-4324-b9c3-0b7dcb0edc5e None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:38.942934 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 583/1184] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:11:38 2022] GET /identity/v3/projects/bece55739d5e4f14bbb05b57f5e7a495 => generated 361 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:38.950099 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3596827-eb3f-4071-9dc9-c0186a1b8015 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:39.063175 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3596827-eb3f-4071-9dc9-c0186a1b8015 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:39.101277 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b9e39-4e9f-44a6-ad51-998ec891b4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:39.101928 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b9e39-4e9f-44a6-ad51-998ec891b4e2 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:39.102208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b9e39-4e9f-44a6-ad51-998ec891b4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:39.102479 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b9e39-4e9f-44a6-ad51-998ec891b4e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:39.103303 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b9e39-4e9f-44a6-ad51-998ec891b4e2 None admin] RBAC: Adding query filter params (role.id=b4c524dd7bb44e94bd3b42d86e1cdf42, scope.project.id=bece55739d5e4f14bbb05b57f5e7a495, user.id=5fb3166f1b4e42bcbd121a394f32ded5) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:39.103613 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b9e39-4e9f-44a6-ad51-998ec891b4e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:39.105088 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b9e39-4e9f-44a6-ad51-998ec891b4e2 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:39.110723 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 602/1185] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:11:38 2022] GET /identity/v3/role_assignments?user.id=5fb3166f1b4e42bcbd121a394f32ded5&scope.project.id=bece55739d5e4f14bbb05b57f5e7a495&role.id=b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:41.253328 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d7fdb-9300-4640-a9de-f42da0cea65c None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:41.253891 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d7fdb-9300-4640-a9de-f42da0cea65c None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:41.253891 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d7fdb-9300-4640-a9de-f42da0cea65c None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:41.255593 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 584/1186] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:11:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:41.264244 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6befece4-01e8-465e-a5e8-c236babb1db7 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:41.264697 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6befece4-01e8-465e-a5e8-c236babb1db7 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:41.264818 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6befece4-01e8-465e-a5e8-c236babb1db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:41.289043 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-6befece4-01e8-465e-a5e8-c236babb1db7 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:41.290249 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-6befece4-01e8-465e-a5e8-c236babb1db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:41.301954 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:41.366446 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 603/1187] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:11:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:41.389904 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d7fdb-9300-4640-a9de-f42da0cea65c None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:41.500432 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d7fdb-9300-4640-a9de-f42da0cea65c None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:41.540374 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68ba483-4fc2-431c-bacc-ae455ccd4f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:41.541127 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68ba483-4fc2-431c-bacc-ae455ccd4f1f None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:41.541404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68ba483-4fc2-431c-bacc-ae455ccd4f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:41.541679 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68ba483-4fc2-431c-bacc-ae455ccd4f1f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:41.552889 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68ba483-4fc2-431c-bacc-ae455ccd4f1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:11:41.560538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-c68ba483-4fc2-431c-bacc-ae455ccd4f1f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:11:41.560538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:41.560538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:41.560538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:41.560538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:41.560538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:41.560538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:41.560538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:41.560538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:41.560538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:41.560538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:41.560538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:41.560538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:11:41.560538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:11:41.560538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:11:41.560538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:11:41.561552 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:11:41.561552 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:11:41.561552 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:11:41.561552 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:11:41.561552 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:41.561552 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:41.561552 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:41.561552 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:41.561552 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:41.561552 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:41.561552 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:41.561552 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:11:41.561552 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:41.561552 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:41.561552 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:41.561552 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:41.562388 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:41.562388 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:41.562388 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:41.562388 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:41.562388 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:11:41.562388 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:11:41.562388 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:11:41.562388 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:11:41.562388 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:11:41.562388 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:11:41.562388 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:11:41.562388 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:11:41.562388 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 585/1188] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 03:11:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:41.568123 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6befece4-01e8-465e-a5e8-c236babb1db7 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:41.671430 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6befece4-01e8-465e-a5e8-c236babb1db7 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:41.707702 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea7805-9ca9-4cbe-bfdc-c36b8edbd771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:41.708368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea7805-9ca9-4cbe-bfdc-c36b8edbd771 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:41.708648 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea7805-9ca9-4cbe-bfdc-c36b8edbd771 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:41.708940 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea7805-9ca9-4cbe-bfdc-c36b8edbd771 None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:41.710086 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea7805-9ca9-4cbe-bfdc-c36b8edbd771 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:41.710554 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea7805-9ca9-4cbe-bfdc-c36b8edbd771 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:41.712711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea7805-9ca9-4cbe-bfdc-c36b8edbd771 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:41.724285 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 604/1189] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:11:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:41.731106 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68ba483-4fc2-431c-bacc-ae455ccd4f1f None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:41.758867 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:41.842378 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68ba483-4fc2-431c-bacc-ae455ccd4f1f None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:41.882140 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ec30e-169c-4880-ac62-38b9cc0c0709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:41.883084 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ec30e-169c-4880-ac62-38b9cc0c0709 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:41.883361 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ec30e-169c-4880-ac62-38b9cc0c0709 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:41.883629 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6ec30e-169c-4880-ac62-38b9cc0c0709 None admin] PATH_INFO: `/v3/users/5fb3166f1b4e42bcbd121a394f32ded5` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:41.884935 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6ec30e-169c-4880-ac62-38b9cc0c0709 None admin] RBAC: Authorizing `identity:get_user(user_id=5fb3166f1b4e42bcbd121a394f32ded5)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:41.886399 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6ec30e-169c-4880-ac62-38b9cc0c0709 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:41.888016 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 586/1190] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:11:41 2022] GET /identity/v3/users/5fb3166f1b4e42bcbd121a394f32ded5 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:41.894425 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea7805-9ca9-4cbe-bfdc-c36b8edbd771 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:41.905978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:42.000651 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea7805-9ca9-4cbe-bfdc-c36b8edbd771 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:42.040451 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a8d76e-baeb-4551-b51b-30c085ce2379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:42.041342 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8d76e-baeb-4551-b51b-30c085ce2379 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:42.041616 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8d76e-baeb-4551-b51b-30c085ce2379 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:42.041886 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8d76e-baeb-4551-b51b-30c085ce2379 None admin] PATH_INFO: `/v3/projects/bece55739d5e4f14bbb05b57f5e7a495` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:42.043177 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a8d76e-baeb-4551-b51b-30c085ce2379 None admin] RBAC: Authorizing `identity:get_project(project_id=bece55739d5e4f14bbb05b57f5e7a495)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:42.044622 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a8d76e-baeb-4551-b51b-30c085ce2379 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:42.046385 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 605/1191] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:11:41 2022] GET /identity/v3/projects/bece55739d5e4f14bbb05b57f5e7a495 => generated 361 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:42.053690 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ec30e-169c-4880-ac62-38b9cc0c0709 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:42.151994 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:42.172249 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6ec30e-169c-4880-ac62-38b9cc0c0709 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:42.209942 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e7a8a-fa4a-4a62-a150-bb4faca9bba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:42.210901 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e7a8a-fa4a-4a62-a150-bb4faca9bba9 None admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:42.211208 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e7a8a-fa4a-4a62-a150-bb4faca9bba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:42.211479 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e7a8a-fa4a-4a62-a150-bb4faca9bba9 None admin] PATH_INFO: `/v3/projects/bece55739d5e4f14bbb05b57f5e7a495/users/5fb3166f1b4e42bcbd121a394f32ded5/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:42.213697 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878e7a8a-fa4a-4a62-a150-bb4faca9bba9 None admin] RBAC: Authorizing `identity:create_grant(project_id=bece55739d5e4f14bbb05b57f5e7a495, user_id=5fb3166f1b4e42bcbd121a394f32ded5, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:42.215127 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878e7a8a-fa4a-4a62-a150-bb4faca9bba9 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:42.232404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 587/1192] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 03:11:42 2022] PUT /identity/v3/projects/bece55739d5e4f14bbb05b57f5e7a495/users/5fb3166f1b4e42bcbd121a394f32ded5/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:11:44.383413 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047fa8b-78a6-4c6a-b4e7-1dce8d4f6999 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:44.384290 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047fa8b-78a6-4c6a-b4e7-1dce8d4f6999 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:44.384290 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047fa8b-78a6-4c6a-b4e7-1dce8d4f6999 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:44.386597 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 606/1193] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:11:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:11:44.392084 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09873383-c036-426f-943f-2ec22261cea3 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:44.392602 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09873383-c036-426f-943f-2ec22261cea3 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:44.392715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09873383-c036-426f-943f-2ec22261cea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:44.411491 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-09873383-c036-426f-943f-2ec22261cea3 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:11:44.412993 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-09873383-c036-426f-943f-2ec22261cea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:11:44.492017 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 588/1194] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:11:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:11:44.526630 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c047fa8b-78a6-4c6a-b4e7-1dce8d4f6999 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:44.623029 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:44.658495 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c047fa8b-78a6-4c6a-b4e7-1dce8d4f6999 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:44.694266 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f86a0e-f5a2-4259-b650-81d1db4d9783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:44.695062 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f86a0e-f5a2-4259-b650-81d1db4d9783 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:44.695341 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f86a0e-f5a2-4259-b650-81d1db4d9783 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:44.695615 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f86a0e-f5a2-4259-b650-81d1db4d9783 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:44.707726 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f86a0e-f5a2-4259-b650-81d1db4d9783 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:11:44.715214 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-b8f86a0e-f5a2-4259-b650-81d1db4d9783 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:11:44.715214 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:11:44.715214 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:11:44.715214 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:11:44.715214 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:11:44.715214 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:11:44.715214 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:11:44.715214 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:11:44.715214 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:11:44.715214 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:11:44.715214 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:11:44.715214 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:11:44.715214 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:11:44.715214 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:11:44.715214 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:11:44.715214 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:11:44.716240 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:11:44.716240 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:11:44.716240 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:11:44.716240 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:11:44.716240 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:11:44.716240 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:11:44.716240 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:11:44.716240 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:11:44.716240 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:11:44.716240 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:11:44.716240 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:11:44.716240 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:11:44.716240 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:11:44.716240 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:11:44.716240 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:11:44.716240 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:11:44.717119 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:11:44.717119 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:11:44.717119 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:11:44.717119 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:11:44.717119 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:11:44.717119 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:11:44.717119 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:11:44.717119 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:11:44.717119 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:11:44.717119 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:11:44.717119 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:11:44.717119 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:11:44.717119 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 607/1195] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 03:11:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:11:44.725018 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09873383-c036-426f-943f-2ec22261cea3 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:44.816213 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:44.848841 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09873383-c036-426f-943f-2ec22261cea3 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:44.883642 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32974ed7-8d0a-467e-b0e0-4d0b4030a49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:44.884234 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32974ed7-8d0a-467e-b0e0-4d0b4030a49c None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:44.884515 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32974ed7-8d0a-467e-b0e0-4d0b4030a49c None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:44.884788 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32974ed7-8d0a-467e-b0e0-4d0b4030a49c None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:44.885578 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32974ed7-8d0a-467e-b0e0-4d0b4030a49c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:44.885912 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32974ed7-8d0a-467e-b0e0-4d0b4030a49c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:44.887293 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32974ed7-8d0a-467e-b0e0-4d0b4030a49c None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:44.895453 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 589/1196] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:11:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:44.901874 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f86a0e-f5a2-4259-b650-81d1db4d9783 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:45.015613 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f86a0e-f5a2-4259-b650-81d1db4d9783 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:45.056363 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a20864-11c1-4569-891b-181a0e4298f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:45.057265 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a20864-11c1-4569-891b-181a0e4298f6 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:45.057548 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a20864-11c1-4569-891b-181a0e4298f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:45.057812 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a20864-11c1-4569-891b-181a0e4298f6 None admin] PATH_INFO: `/v3/users/5fb3166f1b4e42bcbd121a394f32ded5` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:45.059140 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a20864-11c1-4569-891b-181a0e4298f6 None admin] RBAC: Authorizing `identity:get_user(user_id=5fb3166f1b4e42bcbd121a394f32ded5)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:45.060594 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a20864-11c1-4569-891b-181a0e4298f6 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:45.062258 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 608/1197] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 03:11:44 2022] GET /identity/v3/users/5fb3166f1b4e42bcbd121a394f32ded5 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:45.069668 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32974ed7-8d0a-467e-b0e0-4d0b4030a49c None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:45.176004 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32974ed7-8d0a-467e-b0e0-4d0b4030a49c None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:45.215451 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2668c47a-1057-4680-90dc-58f1e3499690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:45.216100 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2668c47a-1057-4680-90dc-58f1e3499690 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:45.216382 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2668c47a-1057-4680-90dc-58f1e3499690 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:45.216657 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2668c47a-1057-4680-90dc-58f1e3499690 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:45.228285 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:45.289817 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2668c47a-1057-4680-90dc-58f1e3499690 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:45.291332 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2668c47a-1057-4680-90dc-58f1e3499690 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:45.415868 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 590/1198] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:11:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:11:45.422288 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a20864-11c1-4569-891b-181a0e4298f6 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:45.426847 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:45.527406 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a20864-11c1-4569-891b-181a0e4298f6 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:45.564044 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b85d78-131a-42a3-8644-43e53b74d551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:45.564971 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b85d78-131a-42a3-8644-43e53b74d551 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:45.565257 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b85d78-131a-42a3-8644-43e53b74d551 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:45.565519 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b85d78-131a-42a3-8644-43e53b74d551 None admin] PATH_INFO: `/v3/projects/bece55739d5e4f14bbb05b57f5e7a495` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:45.566814 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b85d78-131a-42a3-8644-43e53b74d551 None admin] RBAC: Authorizing `identity:get_project(project_id=bece55739d5e4f14bbb05b57f5e7a495)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:45.568274 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b85d78-131a-42a3-8644-43e53b74d551 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:45.569999 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 609/1199] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 03:11:45 2022] GET /identity/v3/projects/bece55739d5e4f14bbb05b57f5e7a495 => generated 361 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:45.576328 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2668c47a-1057-4680-90dc-58f1e3499690 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:11:45.620055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:45.687348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2668c47a-1057-4680-90dc-58f1e3499690 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:11:45.723593 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39133f45-5f24-4693-b29f-43b2ff3f934a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:11:45.724235 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39133f45-5f24-4693-b29f-43b2ff3f934a None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:11:45.724514 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39133f45-5f24-4693-b29f-43b2ff3f934a None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:11:45.724779 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39133f45-5f24-4693-b29f-43b2ff3f934a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:11:45.725631 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39133f45-5f24-4693-b29f-43b2ff3f934a None admin] RBAC: Adding query filter params (role.id=b4c524dd7bb44e94bd3b42d86e1cdf42, scope.project.id=bece55739d5e4f14bbb05b57f5e7a495, user.id=5fb3166f1b4e42bcbd121a394f32ded5) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:11:45.725936 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39133f45-5f24-4693-b29f-43b2ff3f934a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:11:45.727358 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39133f45-5f24-4693-b29f-43b2ff3f934a None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:11:45.733901 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 591/1200] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:11:45 2022] GET /identity/v3/role_assignments?user.id=5fb3166f1b4e42bcbd121a394f32ded5&scope.project.id=bece55739d5e4f14bbb05b57f5e7a495&role.id=b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:11:55.558589 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=64 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:11:55.730645 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=48 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:46.730421 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb51d6c-5442-4a9c-bf5b-f69cf6990711 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:46.731048 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb51d6c-5442-4a9c-bf5b-f69cf6990711 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:46.731048 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb51d6c-5442-4a9c-bf5b-f69cf6990711 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:46.732694 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 610/1201] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:12:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:12:46.740854 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368839fd-84d1-4aae-ba18-30c4005d2418 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:46.740854 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368839fd-84d1-4aae-ba18-30c4005d2418 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:46.740854 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368839fd-84d1-4aae-ba18-30c4005d2418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:46.759713 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-368839fd-84d1-4aae-ba18-30c4005d2418 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:12:46.761332 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-368839fd-84d1-4aae-ba18-30c4005d2418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:12:46.849371 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 592/1202] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:12:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:12:46.876985 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb51d6c-5442-4a9c-bf5b-f69cf6990711 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:46.996312 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb51d6c-5442-4a9c-bf5b-f69cf6990711 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:47.035063 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec36ef-05ec-4173-8095-c9283d44f34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:47.036219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec36ef-05ec-4173-8095-c9283d44f34f None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:47.036699 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec36ef-05ec-4173-8095-c9283d44f34f None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:47.037189 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec36ef-05ec-4173-8095-c9283d44f34f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:47.048496 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ec36ef-05ec-4173-8095-c9283d44f34f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:47.050499 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ec36ef-05ec-4173-8095-c9283d44f34f None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:47.070883 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-b9ec36ef-05ec-4173-8095-c9283d44f34f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:12:47.070883 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:12:47.070883 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:12:47.070883 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:12:47.070883 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:12:47.070883 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:12:47.070883 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:12:47.070883 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:12:47.070883 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:12:47.070883 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:12:47.070883 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:12:47.070883 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:12:47.070883 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:12:47.070883 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:12:47.070883 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:12:47.072035 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:12:47.072035 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:12:47.072035 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:12:47.072035 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:12:47.072035 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:12:47.072035 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:12:47.072035 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:12:47.072035 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:12:47.072035 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:12:47.072035 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:12:47.072035 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:12:47.072035 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:12:47.072035 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:12:47.072035 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:12:47.072035 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:12:47.072035 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:12:47.072035 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:12:47.073464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:12:47.073464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:12:47.073464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:12:47.073464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:12:47.073464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 611/1203] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:12:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:12:47.080136 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368839fd-84d1-4aae-ba18-30c4005d2418 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:47.203432 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368839fd-84d1-4aae-ba18-30c4005d2418 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:47.236751 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:47.245080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4179c1-b07d-4fe3-9b81-8f6fd83fb0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:47.245958 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4179c1-b07d-4fe3-9b81-8f6fd83fb0b1 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:47.246278 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4179c1-b07d-4fe3-9b81-8f6fd83fb0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:47.246709 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4179c1-b07d-4fe3-9b81-8f6fd83fb0b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:47.247463 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4179c1-b07d-4fe3-9b81-8f6fd83fb0b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:12:47.247904 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4179c1-b07d-4fe3-9b81-8f6fd83fb0b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:47.249737 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4179c1-b07d-4fe3-9b81-8f6fd83fb0b1 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:47.262842 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 593/1204] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:47.269437 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec36ef-05ec-4173-8095-c9283d44f34f None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:47.343180 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:47.412692 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec36ef-05ec-4173-8095-c9283d44f34f None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:47.453645 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23edde7d-7d06-4082-a02f-53252c1f3c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:47.454342 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23edde7d-7d06-4082-a02f-53252c1f3c9a None admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:47.454624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23edde7d-7d06-4082-a02f-53252c1f3c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:47.454901 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23edde7d-7d06-4082-a02f-53252c1f3c9a None admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:47.455591 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23edde7d-7d06-4082-a02f-53252c1f3c9a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:47.457185 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23edde7d-7d06-4082-a02f-53252c1f3c9a None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:47.484626 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 612/1205] 10.222.0.50 () {60 vars in 1132 bytes} [Fri May 27 03:12:47 2022] POST /identity/v3/users => generated 251 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:12:49.822075 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8aa8d-4164-4555-af9b-29ed503b171b None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:49.822766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8aa8d-4164-4555-af9b-29ed503b171b None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:49.822766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8aa8d-4164-4555-af9b-29ed503b171b None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:49.823954 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 594/1206] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:12:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:12:49.829651 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8854f-92dd-43d4-a7f2-1bb47f53c4d1 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:49.830116 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8854f-92dd-43d4-a7f2-1bb47f53c4d1 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:49.830218 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8854f-92dd-43d4-a7f2-1bb47f53c4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:49.849589 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-74a8854f-92dd-43d4-a7f2-1bb47f53c4d1 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:12:49.850986 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-74a8854f-92dd-43d4-a7f2-1bb47f53c4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:12:49.923469 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 613/1207] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:12:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:12:49.947376 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8aa8d-4164-4555-af9b-29ed503b171b None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:50.065950 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8aa8d-4164-4555-af9b-29ed503b171b None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:50.101428 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b47e78-b423-49e1-ab38-ca0093088c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:50.102262 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b47e78-b423-49e1-ab38-ca0093088c46 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:50.102615 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b47e78-b423-49e1-ab38-ca0093088c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:50.102988 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b47e78-b423-49e1-ab38-ca0093088c46 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:50.114510 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b47e78-b423-49e1-ab38-ca0093088c46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:12:50.122720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-19b47e78-b423-49e1-ab38-ca0093088c46 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:12:50.122720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:12:50.122720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:12:50.122720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:12:50.122720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:12:50.122720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:12:50.122720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:12:50.122720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:12:50.122720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:12:50.122720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:12:50.122720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:12:50.122720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:12:50.122720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:12:50.122720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:12:50.122720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:12:50.122720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:12:50.123912 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:12:50.123912 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:12:50.123912 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:12:50.123912 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:12:50.123912 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:12:50.123912 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:12:50.123912 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:12:50.123912 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:12:50.123912 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:12:50.123912 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:12:50.123912 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:12:50.123912 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:12:50.123912 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:12:50.123912 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:12:50.123912 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:12:50.123912 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:12:50.124963 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:12:50.124963 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:12:50.124963 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:12:50.124963 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:12:50.124963 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:12:50.124963 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:12:50.124963 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:12:50.124963 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:12:50.124963 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:12:50.124963 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:12:50.124963 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:12:50.124963 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:12:50.126190 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 595/1208] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 03:12:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:12:50.133444 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a8854f-92dd-43d4-a7f2-1bb47f53c4d1 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:50.174727 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:50.239767 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a8854f-92dd-43d4-a7f2-1bb47f53c4d1 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:50.279835 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516801e-f8e0-46e6-a39d-6d9b43fc0808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:50.280825 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516801e-f8e0-46e6-a39d-6d9b43fc0808 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:50.281125 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516801e-f8e0-46e6-a39d-6d9b43fc0808 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:50.281388 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516801e-f8e0-46e6-a39d-6d9b43fc0808 None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:50.284652 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1516801e-f8e0-46e6-a39d-6d9b43fc0808 None admin] RBAC: Adding query filter params (name=service) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:12:50.284652 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1516801e-f8e0-46e6-a39d-6d9b43fc0808 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:50.284652 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1516801e-f8e0-46e6-a39d-6d9b43fc0808 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:50.297845 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 614/1209] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 03:12:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:50.299887 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b47e78-b423-49e1-ab38-ca0093088c46 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:50.359576 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:50.418792 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b47e78-b423-49e1-ab38-ca0093088c46 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:50.456058 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76245a46-ab74-487a-af79-8e4419259c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:50.456900 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76245a46-ab74-487a-af79-8e4419259c7a None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:50.457343 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76245a46-ab74-487a-af79-8e4419259c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:50.457614 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76245a46-ab74-487a-af79-8e4419259c7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:50.468231 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76245a46-ab74-487a-af79-8e4419259c7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:50.471527 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76245a46-ab74-487a-af79-8e4419259c7a None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:50.487771 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-76245a46-ab74-487a-af79-8e4419259c7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:12:50.487771 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:12:50.487771 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:12:50.487771 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:12:50.487771 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:12:50.487771 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:12:50.487771 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:12:50.487771 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:12:50.487771 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:12:50.487771 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:12:50.487771 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:12:50.487771 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:12:50.487771 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:12:50.487771 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:12:50.487771 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:12:50.488766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:12:50.488766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:12:50.488766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:12:50.488766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:12:50.488766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:12:50.488766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:12:50.488766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:12:50.488766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:12:50.488766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:12:50.488766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:12:50.488766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:12:50.488766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:12:50.488766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:12:50.488766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:12:50.488766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:12:50.488766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:12:50.488766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:12:50.489710 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:12:50.489710 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:12:50.489710 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:12:50.489710 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:12:50.489710 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 596/1210] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:12:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:12:50.495162 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516801e-f8e0-46e6-a39d-6d9b43fc0808 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:50.654835 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:50.659028 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516801e-f8e0-46e6-a39d-6d9b43fc0808 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:50.716818 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca600251-b38e-455f-a497-8b58fd873a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:50.717540 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca600251-b38e-455f-a497-8b58fd873a42 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:50.717907 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca600251-b38e-455f-a497-8b58fd873a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:50.718297 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca600251-b38e-455f-a497-8b58fd873a42 None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:50.719056 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca600251-b38e-455f-a497-8b58fd873a42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:12:50.719385 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca600251-b38e-455f-a497-8b58fd873a42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:50.720941 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca600251-b38e-455f-a497-8b58fd873a42 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:50.741002 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 615/1211] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:12:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:12:50.742603 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76245a46-ab74-487a-af79-8e4419259c7a None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:50.859571 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76245a46-ab74-487a-af79-8e4419259c7a None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:50.861364 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:50.903000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea59c416-a997-4d7d-a7ab-e722d458eede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:50.903976 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea59c416-a997-4d7d-a7ab-e722d458eede None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:50.904223 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea59c416-a997-4d7d-a7ab-e722d458eede None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:50.904497 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea59c416-a997-4d7d-a7ab-e722d458eede None admin] PATH_INFO: `/v3/users/placement` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:50.913032 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea59c416-a997-4d7d-a7ab-e722d458eede None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:50.914761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea59c416-a997-4d7d-a7ab-e722d458eede None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:50.932533 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-ea59c416-a997-4d7d-a7ab-e722d458eede None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 03:12:50.932533 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:12:50.932533 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:12:50.932533 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:12:50.932533 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:12:50.932533 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:12:50.932533 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:12:50.932533 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:12:50.932533 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:12:50.932533 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:12:50.932533 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:12:50.932533 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:12:50.932533 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:12:50.932533 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:12:50.932533 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:12:50.932533 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:12:50.933642 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:12:50.933642 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:12:50.933642 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:12:50.933642 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:12:50.933642 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:12:50.933642 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:12:50.933642 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:12:50.933642 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:12:50.933642 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:12:50.933642 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:12:50.933642 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:12:50.933642 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:12:50.933642 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:12:50.933642 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:12:50.933642 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:12:50.933642 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:12:50.933642 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:12:50.933642 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:12:50.935193 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:12:50.935193 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:12:50.935193 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:12:50.935193 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:12:50.935193 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:12:50.935193 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:12:50.935193 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:12:50.935193 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:12:50.935193 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 03:12:50.935193 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:12:50.939154 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 597/1212] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:12:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:12:50.940350 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca600251-b38e-455f-a497-8b58fd873a42 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:51.064872 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca600251-b38e-455f-a497-8b58fd873a42 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:51.100982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0686b365-fd89-4f6a-b584-4686c3cdfc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:51.101600 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686b365-fd89-4f6a-b584-4686c3cdfc25 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:51.103052 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686b365-fd89-4f6a-b584-4686c3cdfc25 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:51.103052 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0686b365-fd89-4f6a-b584-4686c3cdfc25 None admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:51.103052 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0686b365-fd89-4f6a-b584-4686c3cdfc25 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:12:51.103342 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0686b365-fd89-4f6a-b584-4686c3cdfc25 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:51.104740 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0686b365-fd89-4f6a-b584-4686c3cdfc25 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:51.123008 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 616/1213] 10.222.0.50 () {56 vars in 1143 bytes} [Fri May 27 03:12:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:51.129785 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea59c416-a997-4d7d-a7ab-e722d458eede None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:51.242860 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea59c416-a997-4d7d-a7ab-e722d458eede None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:51.283273 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9dafd2-ef0f-4f0b-8804-d82376e28d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:51.284143 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9dafd2-ef0f-4f0b-8804-d82376e28d34 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:51.284680 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9dafd2-ef0f-4f0b-8804-d82376e28d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:51.285102 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9dafd2-ef0f-4f0b-8804-d82376e28d34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:51.320412 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:51.362520 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9dafd2-ef0f-4f0b-8804-d82376e28d34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:51.364027 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9dafd2-ef0f-4f0b-8804-d82376e28d34 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:51.474257 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 598/1214] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:12:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:12:51.480694 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0686b365-fd89-4f6a-b584-4686c3cdfc25 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:51.502789 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:51.598684 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0686b365-fd89-4f6a-b584-4686c3cdfc25 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:51.638323 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ee5095-aa1e-4296-abe6-447c21e725a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:51.639127 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee5095-aa1e-4296-abe6-447c21e725a5 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:51.639425 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee5095-aa1e-4296-abe6-447c21e725a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:51.639694 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee5095-aa1e-4296-abe6-447c21e725a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:51.651438 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ee5095-aa1e-4296-abe6-447c21e725a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:51.653010 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ee5095-aa1e-4296-abe6-447c21e725a5 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:51.672708 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-c0ee5095-aa1e-4296-abe6-447c21e725a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:12:51.672708 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:12:51.672708 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:12:51.672708 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:12:51.672708 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:12:51.672708 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:12:51.672708 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:12:51.672708 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:12:51.672708 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:12:51.672708 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:12:51.672708 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:12:51.672708 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:12:51.672708 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:12:51.672708 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:12:51.672708 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:12:51.675973 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:12:51.675973 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:12:51.675973 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:12:51.675973 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:12:51.675973 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:12:51.675973 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:12:51.675973 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:12:51.675973 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:12:51.675973 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:12:51.675973 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:12:51.675973 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:12:51.675973 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:12:51.675973 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:12:51.675973 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:12:51.675973 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:12:51.675973 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:12:51.675973 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:12:51.677182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:12:51.677182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:12:51.677182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:12:51.677182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:12:51.677182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 617/1215] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:12:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:12:51.680358 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9dafd2-ef0f-4f0b-8804-d82376e28d34 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:51.750575 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:51.792500 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9dafd2-ef0f-4f0b-8804-d82376e28d34 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:51.830186 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee4ffcc-9ea7-4437-91f8-85c0cdf6cc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:51.830876 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4ffcc-9ea7-4437-91f8-85c0cdf6cc37 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:51.831285 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4ffcc-9ea7-4437-91f8-85c0cdf6cc37 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:51.831613 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4ffcc-9ea7-4437-91f8-85c0cdf6cc37 None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:51.832373 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee4ffcc-9ea7-4437-91f8-85c0cdf6cc37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:12:51.832678 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee4ffcc-9ea7-4437-91f8-85c0cdf6cc37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:51.834162 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee4ffcc-9ea7-4437-91f8-85c0cdf6cc37 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:51.847543 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 599/1216] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:12:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:51.854094 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ee5095-aa1e-4296-abe6-447c21e725a5 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:51.946743 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:51.980877 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ee5095-aa1e-4296-abe6-447c21e725a5 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:52.025061 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f27053-e08a-42e4-b2b9-9b38be46eb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:52.026184 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f27053-e08a-42e4-b2b9-9b38be46eb7c None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:52.026464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f27053-e08a-42e4-b2b9-9b38be46eb7c None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:52.026739 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f27053-e08a-42e4-b2b9-9b38be46eb7c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:52.034932 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f27053-e08a-42e4-b2b9-9b38be46eb7c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:52.036425 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f27053-e08a-42e4-b2b9-9b38be46eb7c None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:52.051182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-24f27053-e08a-42e4-b2b9-9b38be46eb7c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:12:52.051182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:12:52.051182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:12:52.051182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:12:52.051182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:12:52.051182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:12:52.051182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:12:52.051182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:12:52.051182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:12:52.051182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:12:52.051182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:12:52.051182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:12:52.051182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:12:52.051182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:12:52.051182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:12:52.052382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:12:52.052382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:12:52.052382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:12:52.052382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:12:52.052382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:12:52.052382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:12:52.052382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:12:52.052382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:12:52.052382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:12:52.052382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:12:52.052382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:12:52.052382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:12:52.052382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:12:52.052382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:12:52.052382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:12:52.052382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:12:52.052382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:12:52.058124 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:12:52.058124 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:12:52.058124 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:12:52.058124 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:12:52.058124 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:12:52.058124 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:12:52.058124 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:12:52.058124 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:12:52.058124 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 618/1217] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:12:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:12:52.058719 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee4ffcc-9ea7-4437-91f8-85c0cdf6cc37 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:52.171476 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee4ffcc-9ea7-4437-91f8-85c0cdf6cc37 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:52.196053 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:52.209557 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2554de4e-60cf-4410-bbe7-a785a21ecaee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:52.210360 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554de4e-60cf-4410-bbe7-a785a21ecaee None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:52.210731 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554de4e-60cf-4410-bbe7-a785a21ecaee None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:52.211096 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554de4e-60cf-4410-bbe7-a785a21ecaee None admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:52.211974 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2554de4e-60cf-4410-bbe7-a785a21ecaee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:12:52.212377 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2554de4e-60cf-4410-bbe7-a785a21ecaee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:52.213952 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2554de4e-60cf-4410-bbe7-a785a21ecaee None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:52.234658 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 600/1218] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 03:12:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:12:52.235964 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f27053-e08a-42e4-b2b9-9b38be46eb7c None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:52.366530 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f27053-e08a-42e4-b2b9-9b38be46eb7c None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:52.397977 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:52.404301 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7650df81-ed30-4488-b5b5-9ec1346652e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:52.405044 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7650df81-ed30-4488-b5b5-9ec1346652e9 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:52.405812 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7650df81-ed30-4488-b5b5-9ec1346652e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:52.406368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7650df81-ed30-4488-b5b5-9ec1346652e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:52.407354 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7650df81-ed30-4488-b5b5-9ec1346652e9 None admin] RBAC: Adding query filter params (role.id=3451aa51e64642768b2228126d20fa03, scope.project.id=f8398941252346468483ea0c863122f4, user.id=30963f2f464e4f54a8aea0175370a235) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:12:52.407663 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7650df81-ed30-4488-b5b5-9ec1346652e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:52.409179 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7650df81-ed30-4488-b5b5-9ec1346652e9 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:52.415271 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 619/1219] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:12:52 2022] GET /identity/v3/role_assignments?user.id=30963f2f464e4f54a8aea0175370a235&scope.project.id=f8398941252346468483ea0c863122f4&role.id=3451aa51e64642768b2228126d20fa03 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:54.675602 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c6c6b3-8d57-4e38-9ea6-89c0a229705e None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:54.676155 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c6c6b3-8d57-4e38-9ea6-89c0a229705e None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:54.676155 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c6c6b3-8d57-4e38-9ea6-89c0a229705e None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:54.677740 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 601/1220] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:12:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:12:54.683699 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151eb5b-8c09-420d-af26-92f1f2d536d4 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:54.684146 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151eb5b-8c09-420d-af26-92f1f2d536d4 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:54.684318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151eb5b-8c09-420d-af26-92f1f2d536d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:54.701740 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-5151eb5b-8c09-420d-af26-92f1f2d536d4 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:12:54.702913 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-5151eb5b-8c09-420d-af26-92f1f2d536d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:12:54.773738 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 620/1221] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:12:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:12:54.796566 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c6c6b3-8d57-4e38-9ea6-89c0a229705e None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:54.906569 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c6c6b3-8d57-4e38-9ea6-89c0a229705e None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:54.944764 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c91ef21-077e-4e18-9ac4-e622a387e0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:54.945463 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91ef21-077e-4e18-9ac4-e622a387e0b8 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:54.945742 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91ef21-077e-4e18-9ac4-e622a387e0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:54.946012 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91ef21-077e-4e18-9ac4-e622a387e0b8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:54.956765 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c91ef21-077e-4e18-9ac4-e622a387e0b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:12:54.964475 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-6c91ef21-077e-4e18-9ac4-e622a387e0b8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:12:54.964475 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:12:54.964475 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:12:54.964475 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:12:54.964475 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:12:54.964475 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:12:54.964475 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:12:54.964475 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:12:54.964475 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:12:54.964475 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:12:54.964475 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:12:54.964475 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:12:54.964475 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:12:54.964475 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:12:54.964475 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:12:54.964475 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:12:54.965714 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:12:54.965714 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:12:54.965714 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:12:54.965714 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:12:54.965714 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:12:54.965714 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:12:54.965714 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:12:54.965714 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:12:54.965714 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:12:54.965714 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:12:54.965714 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:12:54.965714 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:12:54.965714 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:12:54.965714 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:12:54.965714 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:12:54.965714 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:12:54.966566 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:12:54.966566 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:12:54.966566 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:12:54.966566 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:12:54.966566 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:12:54.966566 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:12:54.966566 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:12:54.966566 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:12:54.966566 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:12:54.966566 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:12:54.966566 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:12:54.966566 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:12:54.966566 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 602/1222] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 03:12:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:12:54.972504 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151eb5b-8c09-420d-af26-92f1f2d536d4 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:55.088775 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151eb5b-8c09-420d-af26-92f1f2d536d4 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:55.117602 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:55.130890 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745bb34-fa95-48e4-b927-be8c1cec2d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:55.131705 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745bb34-fa95-48e4-b927-be8c1cec2d38 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:55.131985 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745bb34-fa95-48e4-b927-be8c1cec2d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:55.132339 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745bb34-fa95-48e4-b927-be8c1cec2d38 None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:55.133187 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8745bb34-fa95-48e4-b927-be8c1cec2d38 None admin] RBAC: Adding query filter params (name=service) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:12:55.133486 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8745bb34-fa95-48e4-b927-be8c1cec2d38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:55.134863 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8745bb34-fa95-48e4-b927-be8c1cec2d38 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:55.143269 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 621/1223] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 03:12:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:55.149545 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c91ef21-077e-4e18-9ac4-e622a387e0b8 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:55.199032 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:55.260671 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c91ef21-077e-4e18-9ac4-e622a387e0b8 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:55.295374 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2508bb41-d228-485f-b2ce-65d4064bfbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:55.296223 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508bb41-d228-485f-b2ce-65d4064bfbf8 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:55.296509 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508bb41-d228-485f-b2ce-65d4064bfbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:55.296807 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508bb41-d228-485f-b2ce-65d4064bfbf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:55.307966 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2508bb41-d228-485f-b2ce-65d4064bfbf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:55.309558 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2508bb41-d228-485f-b2ce-65d4064bfbf8 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:55.327149 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-2508bb41-d228-485f-b2ce-65d4064bfbf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:12:55.327149 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:12:55.327149 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:12:55.327149 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:12:55.327149 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:12:55.327149 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:12:55.327149 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:12:55.327149 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:12:55.327149 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:12:55.327149 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:12:55.327149 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:12:55.327149 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:12:55.327149 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:12:55.327149 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:12:55.327149 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:12:55.328257 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:12:55.328257 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:12:55.328257 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:12:55.328257 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:12:55.328257 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:12:55.328257 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:12:55.328257 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:12:55.328257 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:12:55.328257 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:12:55.328257 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:12:55.328257 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:12:55.328257 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:12:55.328257 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:12:55.328257 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:12:55.328257 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:12:55.328257 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:12:55.328257 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:12:55.329198 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:12:55.329198 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:12:55.329198 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:12:55.329198 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:12:55.329198 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 603/1224] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:12:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:12:55.334386 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745bb34-fa95-48e4-b927-be8c1cec2d38 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:55.443568 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745bb34-fa95-48e4-b927-be8c1cec2d38 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:55.488992 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ca92a-81b2-4e42-b2fd-2766cc753022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:55.489593 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ca92a-81b2-4e42-b2fd-2766cc753022 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:55.489877 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ca92a-81b2-4e42-b2fd-2766cc753022 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:55.490211 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765ca92a-81b2-4e42-b2fd-2766cc753022 None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:55.491007 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765ca92a-81b2-4e42-b2fd-2766cc753022 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:12:55.491323 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765ca92a-81b2-4e42-b2fd-2766cc753022 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:55.492889 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765ca92a-81b2-4e42-b2fd-2766cc753022 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:55.507512 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 622/1225] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:12:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:55.513568 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2508bb41-d228-485f-b2ce-65d4064bfbf8 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:55.629146 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:55.636465 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2508bb41-d228-485f-b2ce-65d4064bfbf8 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:55.674078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837510a7-7b86-4d9e-bffc-31f533ad436f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:55.674999 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837510a7-7b86-4d9e-bffc-31f533ad436f None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:55.675288 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837510a7-7b86-4d9e-bffc-31f533ad436f None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:55.675691 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837510a7-7b86-4d9e-bffc-31f533ad436f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:55.683522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837510a7-7b86-4d9e-bffc-31f533ad436f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:55.685133 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837510a7-7b86-4d9e-bffc-31f533ad436f None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:55.699400 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-837510a7-7b86-4d9e-bffc-31f533ad436f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 03:12:55.699400 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:12:55.699400 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:12:55.699400 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:12:55.699400 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:12:55.699400 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:12:55.699400 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:12:55.699400 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:12:55.699400 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:12:55.699400 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:12:55.699400 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:12:55.699400 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:12:55.699400 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:12:55.699400 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:12:55.699400 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:12:55.699400 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:12:55.700636 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:12:55.700636 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:12:55.700636 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:12:55.700636 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:12:55.700636 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:12:55.700636 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:12:55.700636 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:12:55.700636 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:12:55.700636 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:12:55.700636 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:12:55.700636 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:12:55.700636 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:12:55.700636 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:12:55.700636 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:12:55.700636 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:12:55.700636 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:12:55.700636 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:12:55.700636 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:12:55.701834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:12:55.701834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:12:55.701834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:12:55.701834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:12:55.701834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:12:55.701834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:12:55.701834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:12:55.701834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:12:55.701834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 03:12:55.701834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:12:55.701834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 604/1226] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:12:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:12:55.706617 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ca92a-81b2-4e42-b2fd-2766cc753022 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:55.755656 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:55.812366 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765ca92a-81b2-4e42-b2fd-2766cc753022 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:55.858373 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b4c2b-5c1e-4a68-9890-96da0924315c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:55.859119 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b4c2b-5c1e-4a68-9890-96da0924315c None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:55.859413 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b4c2b-5c1e-4a68-9890-96da0924315c None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:55.859705 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b4c2b-5c1e-4a68-9890-96da0924315c None admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:55.860567 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78b4c2b-5c1e-4a68-9890-96da0924315c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:12:55.860906 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78b4c2b-5c1e-4a68-9890-96da0924315c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:55.862636 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78b4c2b-5c1e-4a68-9890-96da0924315c None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:55.883921 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 623/1227] 10.222.0.50 () {56 vars in 1143 bytes} [Fri May 27 03:12:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:55.892239 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837510a7-7b86-4d9e-bffc-31f533ad436f None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:56.009215 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837510a7-7b86-4d9e-bffc-31f533ad436f None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:56.053130 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc9c25e-985f-49b2-b2b3-9848b2a1dbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:56.054119 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc9c25e-985f-49b2-b2b3-9848b2a1dbf1 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:56.054444 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc9c25e-985f-49b2-b2b3-9848b2a1dbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:56.054739 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc9c25e-985f-49b2-b2b3-9848b2a1dbf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:56.067067 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc9c25e-985f-49b2-b2b3-9848b2a1dbf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:56.068753 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc9c25e-985f-49b2-b2b3-9848b2a1dbf1 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:56.089479 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-ddc9c25e-985f-49b2-b2b3-9848b2a1dbf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:12:56.089479 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:12:56.089479 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:12:56.089479 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:12:56.089479 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:12:56.089479 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:12:56.089479 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:12:56.089479 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:12:56.089479 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:12:56.089479 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:12:56.089479 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:12:56.089479 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:12:56.089479 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:12:56.089479 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:12:56.089479 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:12:56.090664 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:12:56.090664 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:12:56.090664 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:12:56.090664 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:12:56.090664 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:12:56.090664 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:12:56.090664 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:12:56.090664 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:12:56.090664 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:12:56.090664 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:12:56.090664 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:12:56.090664 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:12:56.090664 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:12:56.090664 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:12:56.090664 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:12:56.090664 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:12:56.090664 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:12:56.096463 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:12:56.096463 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:12:56.096463 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:12:56.096463 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:12:56.096463 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 605/1228] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:12:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:12:56.097125 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b4c2b-5c1e-4a68-9890-96da0924315c None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:56.204850 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b4c2b-5c1e-4a68-9890-96da0924315c None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:56.207871 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:56.244473 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae20ae42-ce95-48fe-b205-92370455df1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:56.245214 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae20ae42-ce95-48fe-b205-92370455df1b None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:56.245485 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae20ae42-ce95-48fe-b205-92370455df1b None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:56.245821 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae20ae42-ce95-48fe-b205-92370455df1b None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:56.246571 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae20ae42-ce95-48fe-b205-92370455df1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:12:56.246924 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae20ae42-ce95-48fe-b205-92370455df1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:56.248312 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae20ae42-ce95-48fe-b205-92370455df1b None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:56.261394 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 624/1229] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:12:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:12:56.266793 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc9c25e-985f-49b2-b2b3-9848b2a1dbf1 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:56.277442 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:56.374776 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc9c25e-985f-49b2-b2b3-9848b2a1dbf1 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:56.410914 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c2d214-182f-4cb7-bc1d-f5151f92e19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:56.411874 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2d214-182f-4cb7-bc1d-f5151f92e19a None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:56.412214 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2d214-182f-4cb7-bc1d-f5151f92e19a None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:56.412604 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2d214-182f-4cb7-bc1d-f5151f92e19a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:56.419826 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c2d214-182f-4cb7-bc1d-f5151f92e19a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:56.421308 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c2d214-182f-4cb7-bc1d-f5151f92e19a None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:56.439567 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-e3c2d214-182f-4cb7-bc1d-f5151f92e19a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:12:56.439567 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:12:56.439567 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:12:56.439567 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:12:56.439567 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:12:56.439567 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:12:56.439567 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:12:56.439567 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:12:56.439567 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:12:56.439567 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:12:56.439567 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:12:56.439567 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:12:56.439567 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:12:56.439567 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:12:56.439567 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:12:56.441030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:12:56.441030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:12:56.441030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:12:56.441030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:12:56.441030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:12:56.441030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:12:56.441030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:12:56.441030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:12:56.441030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:12:56.441030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:12:56.441030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:12:56.441030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:12:56.441030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:12:56.441030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:12:56.441030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:12:56.441030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:12:56.441030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:12:56.442243 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:12:56.442243 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:12:56.442243 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:12:56.442243 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:12:56.442243 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:12:56.442243 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:12:56.442243 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:12:56.442243 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:12:56.442243 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 606/1230] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:12:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:12:56.448030 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae20ae42-ce95-48fe-b205-92370455df1b None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:56.566378 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae20ae42-ce95-48fe-b205-92370455df1b None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:56.604084 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80050a-4c24-453c-9bd5-778a5a2fa0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:56.604835 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80050a-4c24-453c-9bd5-778a5a2fa0da None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:56.605159 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80050a-4c24-453c-9bd5-778a5a2fa0da None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:56.605448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80050a-4c24-453c-9bd5-778a5a2fa0da None admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:56.606383 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f80050a-4c24-453c-9bd5-778a5a2fa0da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:12:56.606696 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f80050a-4c24-453c-9bd5-778a5a2fa0da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:56.608247 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f80050a-4c24-453c-9bd5-778a5a2fa0da None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:56.622134 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 625/1231] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 03:12:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:56.628987 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c2d214-182f-4cb7-bc1d-f5151f92e19a None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:56.701950 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:56.739768 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c2d214-182f-4cb7-bc1d-f5151f92e19a None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:56.776165 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe37549c-7de3-4e43-9d94-8b178b762cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:56.777109 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37549c-7de3-4e43-9d94-8b178b762cc2 None admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:56.777379 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37549c-7de3-4e43-9d94-8b178b762cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:56.777650 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37549c-7de3-4e43-9d94-8b178b762cc2 None admin] PATH_INFO: `/v3/projects/f8398941252346468483ea0c863122f4/users/30963f2f464e4f54a8aea0175370a235/roles/3451aa51e64642768b2228126d20fa03` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:56.793490 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe37549c-7de3-4e43-9d94-8b178b762cc2 None admin] RBAC: Authorizing `identity:create_grant(project_id=f8398941252346468483ea0c863122f4, user_id=30963f2f464e4f54a8aea0175370a235, role_id=3451aa51e64642768b2228126d20fa03)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:56.794974 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe37549c-7de3-4e43-9d94-8b178b762cc2 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:56.809028 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 607/1232] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 03:12:56 2022] PUT /identity/v3/projects/f8398941252346468483ea0c863122f4/users/30963f2f464e4f54a8aea0175370a235/roles/3451aa51e64642768b2228126d20fa03 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:12:59.130664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf55c2-07dd-485a-ac80-753a8528df24 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:59.131224 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf55c2-07dd-485a-ac80-753a8528df24 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:59.131224 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf55c2-07dd-485a-ac80-753a8528df24 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:59.132885 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 626/1233] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:12:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:12:59.138362 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de643ae0-a5b9-4b86-bbb2-082018731946 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:59.138793 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de643ae0-a5b9-4b86-bbb2-082018731946 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:59.138897 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de643ae0-a5b9-4b86-bbb2-082018731946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:59.156764 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-de643ae0-a5b9-4b86-bbb2-082018731946 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:12:59.158009 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-de643ae0-a5b9-4b86-bbb2-082018731946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:12:59.227963 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 608/1234] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:12:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:12:59.254313 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf55c2-07dd-485a-ac80-753a8528df24 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:59.282060 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:59.364032 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf55c2-07dd-485a-ac80-753a8528df24 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:59.399905 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60d45a-f36f-4658-8fa9-c060d273588e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:59.400649 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60d45a-f36f-4658-8fa9-c060d273588e None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:59.400920 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60d45a-f36f-4658-8fa9-c060d273588e None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:59.401217 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c60d45a-f36f-4658-8fa9-c060d273588e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:59.412871 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c60d45a-f36f-4658-8fa9-c060d273588e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:12:59.420502 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-4c60d45a-f36f-4658-8fa9-c060d273588e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 03:12:59.420502 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:12:59.420502 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:12:59.420502 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:12:59.420502 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:12:59.420502 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:12:59.420502 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:12:59.420502 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:12:59.420502 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:12:59.420502 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:12:59.420502 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:12:59.420502 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:12:59.420502 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:12:59.420502 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:12:59.420502 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:12:59.420502 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:12:59.421590 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:12:59.421590 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:12:59.421590 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:12:59.421590 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:12:59.421590 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:12:59.421590 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:12:59.421590 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:12:59.421590 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:12:59.421590 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:12:59.421590 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:12:59.421590 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:12:59.421590 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:12:59.421590 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:12:59.421590 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:12:59.421590 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:12:59.421590 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:12:59.422455 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:12:59.422455 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:12:59.422455 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:12:59.422455 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:12:59.422455 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:12:59.422455 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:12:59.422455 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:12:59.422455 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:12:59.422455 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:12:59.422455 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:12:59.422455 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 03:12:59.422455 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:12:59.422455 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 627/1235] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 03:12:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:12:59.428317 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de643ae0-a5b9-4b86-bbb2-082018731946 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:59.474725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:59.531767 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de643ae0-a5b9-4b86-bbb2-082018731946 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:59.568885 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca386b-4f41-486c-9337-8c4dc516c467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:59.569599 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca386b-4f41-486c-9337-8c4dc516c467 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:59.569880 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca386b-4f41-486c-9337-8c4dc516c467 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:59.570158 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca386b-4f41-486c-9337-8c4dc516c467 None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:59.570918 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ca386b-4f41-486c-9337-8c4dc516c467 None admin] RBAC: Adding query filter params (name=service) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:12:59.571225 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ca386b-4f41-486c-9337-8c4dc516c467 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:59.572625 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ca386b-4f41-486c-9337-8c4dc516c467 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:59.584309 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 609/1236] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 03:12:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:12:59.590383 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60d45a-f36f-4658-8fa9-c060d273588e None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:59.680943 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:59.697706 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c60d45a-f36f-4658-8fa9-c060d273588e None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:59.731993 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab6970-b68a-47f9-a4ed-1a3f281b97d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:59.732803 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab6970-b68a-47f9-a4ed-1a3f281b97d3 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:59.733114 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab6970-b68a-47f9-a4ed-1a3f281b97d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:59.733390 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab6970-b68a-47f9-a4ed-1a3f281b97d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:59.744381 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab6970-b68a-47f9-a4ed-1a3f281b97d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:59.745918 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab6970-b68a-47f9-a4ed-1a3f281b97d3 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:59.763930 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-96ab6970-b68a-47f9-a4ed-1a3f281b97d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:12:59.763930 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:12:59.763930 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:12:59.763930 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:12:59.763930 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:12:59.763930 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:12:59.763930 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:12:59.763930 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:12:59.763930 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:12:59.763930 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:12:59.763930 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:12:59.763930 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:12:59.763930 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:12:59.763930 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:12:59.763930 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:12:59.764995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:12:59.764995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:12:59.764995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:12:59.764995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:12:59.764995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:12:59.764995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:12:59.764995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:12:59.764995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:12:59.764995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:12:59.764995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:12:59.764995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:12:59.764995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:12:59.764995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:12:59.764995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:12:59.764995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:12:59.764995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:12:59.764995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:12:59.765962 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:12:59.765962 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:12:59.765962 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:12:59.765962 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:12:59.765962 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 628/1237] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:12:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:12:59.771171 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca386b-4f41-486c-9337-8c4dc516c467 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:12:59.878230 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca386b-4f41-486c-9337-8c4dc516c467 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:12:59.887480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:12:59.916526 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa1584-1720-457c-baa7-28131c492640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:12:59.917228 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa1584-1720-457c-baa7-28131c492640 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:12:59.917517 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa1584-1720-457c-baa7-28131c492640 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:12:59.917782 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efa1584-1720-457c-baa7-28131c492640 None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:12:59.918509 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efa1584-1720-457c-baa7-28131c492640 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:12:59.918826 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efa1584-1720-457c-baa7-28131c492640 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:12:59.920220 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efa1584-1720-457c-baa7-28131c492640 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:12:59.932674 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 610/1238] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:12:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:12:59.938926 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab6970-b68a-47f9-a4ed-1a3f281b97d3 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:00.061888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab6970-b68a-47f9-a4ed-1a3f281b97d3 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:00.100707 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23461d55-75c9-4d41-9f54-864217de1833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:00.101548 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23461d55-75c9-4d41-9f54-864217de1833 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:00.101841 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23461d55-75c9-4d41-9f54-864217de1833 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:00.102105 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23461d55-75c9-4d41-9f54-864217de1833 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:00.105250 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:00.111780 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23461d55-75c9-4d41-9f54-864217de1833 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:00.113650 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23461d55-75c9-4d41-9f54-864217de1833 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:00.129372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-23461d55-75c9-4d41-9f54-864217de1833 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 03:13:00.129372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:00.129372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:00.129372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:00.129372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:00.129372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:00.129372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:00.129372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:00.129372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:00.129372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:00.129372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:00.129372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:00.129372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:13:00.129372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:13:00.129372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:13:00.129372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:13:00.130575 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:00.130575 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:00.130575 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:13:00.130575 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:13:00.130575 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:13:00.130575 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:13:00.130575 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:00.130575 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:00.130575 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:00.130575 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:00.130575 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:00.130575 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:13:00.130575 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:00.130575 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:00.130575 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:00.130575 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:00.130575 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:00.130575 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:00.132137 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:00.132137 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:00.132137 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:13:00.132137 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:13:00.132137 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:13:00.132137 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:13:00.132137 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:13:00.132137 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:13:00.132137 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 03:13:00.132137 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:13:00.132137 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 629/1239] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:12:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:13:00.136872 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa1584-1720-457c-baa7-28131c492640 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:00.246899 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efa1584-1720-457c-baa7-28131c492640 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:00.282199 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6556651-7a8d-4eec-aba0-424ec1b0ab13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:00.282878 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6556651-7a8d-4eec-aba0-424ec1b0ab13 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:00.283157 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6556651-7a8d-4eec-aba0-424ec1b0ab13 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:00.283430 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6556651-7a8d-4eec-aba0-424ec1b0ab13 None admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:00.284420 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6556651-7a8d-4eec-aba0-424ec1b0ab13 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:00.284730 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6556651-7a8d-4eec-aba0-424ec1b0ab13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:00.286334 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6556651-7a8d-4eec-aba0-424ec1b0ab13 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:00.306079 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 611/1240] 10.222.0.50 () {56 vars in 1143 bytes} [Fri May 27 03:13:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:00.311879 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23461d55-75c9-4d41-9f54-864217de1833 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:00.423353 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23461d55-75c9-4d41-9f54-864217de1833 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:00.459532 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aed1e7-30b3-491e-b9ac-4cde6b3a5353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:00.460315 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aed1e7-30b3-491e-b9ac-4cde6b3a5353 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:00.460596 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aed1e7-30b3-491e-b9ac-4cde6b3a5353 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:00.460869 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aed1e7-30b3-491e-b9ac-4cde6b3a5353 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:00.536891 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:00.544984 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aed1e7-30b3-491e-b9ac-4cde6b3a5353 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:00.546595 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aed1e7-30b3-491e-b9ac-4cde6b3a5353 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:00.667620 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 630/1241] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:13:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 03:13:00.669102 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6556651-7a8d-4eec-aba0-424ec1b0ab13 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:00.699039 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:00.778459 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6556651-7a8d-4eec-aba0-424ec1b0ab13 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:00.818716 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf9c00a-0f00-473a-9fdb-b355fcbc3b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:00.819447 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9c00a-0f00-473a-9fdb-b355fcbc3b2a None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:00.819729 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9c00a-0f00-473a-9fdb-b355fcbc3b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:00.819999 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9c00a-0f00-473a-9fdb-b355fcbc3b2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:00.831729 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf9c00a-0f00-473a-9fdb-b355fcbc3b2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:00.833219 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf9c00a-0f00-473a-9fdb-b355fcbc3b2a None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:00.852700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-9cf9c00a-0f00-473a-9fdb-b355fcbc3b2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:13:00.852700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:00.852700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:00.852700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:00.852700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:00.852700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:00.852700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:00.852700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:00.852700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:00.852700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:00.852700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:00.852700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:00.852700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:13:00.852700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:13:00.852700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:13:00.854618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:13:00.854618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:00.854618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:00.854618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:00.854618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:00.854618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:00.854618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:00.854618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:00.854618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:13:00.854618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:00.854618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:00.854618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:00.854618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:00.854618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:00.854618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:00.854618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:00.854618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:00.855717 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:13:00.855717 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:13:00.855717 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:13:00.855717 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:13:00.855717 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 612/1242] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:13:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:13:00.860151 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aed1e7-30b3-491e-b9ac-4cde6b3a5353 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:00.966570 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:00.973310 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aed1e7-30b3-491e-b9ac-4cde6b3a5353 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:01.017597 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79315285-66f7-4d16-a51d-092d3d921fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:01.018378 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79315285-66f7-4d16-a51d-092d3d921fea None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:01.018651 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79315285-66f7-4d16-a51d-092d3d921fea None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:01.018925 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79315285-66f7-4d16-a51d-092d3d921fea None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:01.019753 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79315285-66f7-4d16-a51d-092d3d921fea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:01.020034 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79315285-66f7-4d16-a51d-092d3d921fea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:01.021468 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79315285-66f7-4d16-a51d-092d3d921fea None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:01.036243 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 631/1243] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:13:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:01.042830 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf9c00a-0f00-473a-9fdb-b355fcbc3b2a None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:01.140413 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:01.166370 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf9c00a-0f00-473a-9fdb-b355fcbc3b2a None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:01.204999 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c38dc-512d-4a1c-8c5f-757d20467443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:01.206223 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c38dc-512d-4a1c-8c5f-757d20467443 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:01.206718 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c38dc-512d-4a1c-8c5f-757d20467443 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:01.207028 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c38dc-512d-4a1c-8c5f-757d20467443 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:01.214882 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c38dc-512d-4a1c-8c5f-757d20467443 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:01.216642 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c38dc-512d-4a1c-8c5f-757d20467443 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:01.232568 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-784c38dc-512d-4a1c-8c5f-757d20467443 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:13:01.232568 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:01.232568 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:01.232568 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:01.232568 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:01.232568 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:01.232568 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:01.232568 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:01.232568 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:01.232568 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:01.232568 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:01.232568 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:01.232568 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:13:01.232568 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:13:01.232568 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:13:01.233573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:13:01.233573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:01.233573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:01.233573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:01.233573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:01.233573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:01.233573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:01.233573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:01.233573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:13:01.233573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:01.233573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:01.233573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:01.233573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:01.233573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:01.233573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:01.233573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:01.233573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:01.234477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:13:01.234477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:13:01.234477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:13:01.234477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:13:01.234477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:13:01.234477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:13:01.234477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:13:01.234477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:13:01.234477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 613/1244] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:13:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:13:01.240202 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79315285-66f7-4d16-a51d-092d3d921fea None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:01.348301 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79315285-66f7-4d16-a51d-092d3d921fea None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:01.384533 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3c5121-281e-4527-b1b6-c6125f2073d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:01.385234 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c5121-281e-4527-b1b6-c6125f2073d5 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:01.385517 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c5121-281e-4527-b1b6-c6125f2073d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:01.385793 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c5121-281e-4527-b1b6-c6125f2073d5 None admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:01.386548 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3c5121-281e-4527-b1b6-c6125f2073d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:01.386857 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3c5121-281e-4527-b1b6-c6125f2073d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:01.388312 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3c5121-281e-4527-b1b6-c6125f2073d5 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:01.402644 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 632/1245] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 03:13:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:01.410816 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c38dc-512d-4a1c-8c5f-757d20467443 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:01.522340 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c38dc-512d-4a1c-8c5f-757d20467443 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:01.559381 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b1c84-9284-4fdd-aced-5a1201765da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:01.560044 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b1c84-9284-4fdd-aced-5a1201765da3 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:01.560313 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b1c84-9284-4fdd-aced-5a1201765da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:01.560621 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b1c84-9284-4fdd-aced-5a1201765da3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:01.561481 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5b1c84-9284-4fdd-aced-5a1201765da3 None admin] RBAC: Adding query filter params (role.id=3451aa51e64642768b2228126d20fa03, scope.project.id=f8398941252346468483ea0c863122f4, user.id=30963f2f464e4f54a8aea0175370a235) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:01.561785 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5b1c84-9284-4fdd-aced-5a1201765da3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:01.563247 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5b1c84-9284-4fdd-aced-5a1201765da3 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:01.569684 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 614/1246] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:13:01 2022] GET /identity/v3/role_assignments?user.id=30963f2f464e4f54a8aea0175370a235&scope.project.id=f8398941252346468483ea0c863122f4&role.id=3451aa51e64642768b2228126d20fa03 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:04.069290 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d38b8e-2c67-45a6-bc97-5e8e2c5f3507 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:04.069885 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d38b8e-2c67-45a6-bc97-5e8e2c5f3507 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:04.069885 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d38b8e-2c67-45a6-bc97-5e8e2c5f3507 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:04.071390 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 633/1247] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:13:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:13:04.077987 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59359c-516c-4519-85ca-9214a5a686ed None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:04.078438 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59359c-516c-4519-85ca-9214a5a686ed None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:04.078584 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59359c-516c-4519-85ca-9214a5a686ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:04.085111 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:04.098985 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-fd59359c-516c-4519-85ca-9214a5a686ed None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:13:04.100204 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-fd59359c-516c-4519-85ca-9214a5a686ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:13:04.174941 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 615/1248] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:13:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 03:13:04.207268 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d38b8e-2c67-45a6-bc97-5e8e2c5f3507 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:04.229428 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:04.331460 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d38b8e-2c67-45a6-bc97-5e8e2c5f3507 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:04.374002 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623bb257-e4ef-456d-b223-4f18711b0c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:04.374883 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623bb257-e4ef-456d-b223-4f18711b0c53 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:04.375190 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623bb257-e4ef-456d-b223-4f18711b0c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:04.375564 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623bb257-e4ef-456d-b223-4f18711b0c53 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:04.387411 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623bb257-e4ef-456d-b223-4f18711b0c53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:13:04.396254 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-623bb257-e4ef-456d-b223-4f18711b0c53 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:13:04.396254 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:04.396254 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:04.396254 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:04.396254 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:04.396254 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:04.396254 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:04.396254 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:04.396254 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:04.396254 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:04.396254 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:04.396254 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:04.396254 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:13:04.396254 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:13:04.396254 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:13:04.396254 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:13:04.397420 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:13:04.397420 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:13:04.397420 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:13:04.397420 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:13:04.397420 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:04.397420 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:04.397420 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:04.397420 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:04.397420 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:04.397420 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:04.397420 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:04.397420 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:13:04.397420 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:04.397420 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:04.397420 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:04.397420 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:04.401780 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:04.401780 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:04.401780 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:04.401780 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:04.401780 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:13:04.401780 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:13:04.401780 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:13:04.401780 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:13:04.401780 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:13:04.401780 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:13:04.401780 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:13:04.401780 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:13:04.401780 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 634/1249] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 03:13:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:13:04.405410 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59359c-516c-4519-85ca-9214a5a686ed None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:04.523093 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59359c-516c-4519-85ca-9214a5a686ed None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:04.557210 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:04.563093 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab81abf-4762-4abc-834d-0b68dc82e4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:04.563868 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab81abf-4762-4abc-834d-0b68dc82e4cf None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:04.564313 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab81abf-4762-4abc-834d-0b68dc82e4cf None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:04.564598 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab81abf-4762-4abc-834d-0b68dc82e4cf None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:04.565824 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab81abf-4762-4abc-834d-0b68dc82e4cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:04.566190 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab81abf-4762-4abc-834d-0b68dc82e4cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:04.567895 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab81abf-4762-4abc-834d-0b68dc82e4cf None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:04.583532 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 616/1250] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:13:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:04.584690 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623bb257-e4ef-456d-b223-4f18711b0c53 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:04.665069 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:04.689856 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623bb257-e4ef-456d-b223-4f18711b0c53 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:04.728327 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79117bd7-520e-4780-8e45-21ac605a7c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:04.729162 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79117bd7-520e-4780-8e45-21ac605a7c07 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:04.729437 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79117bd7-520e-4780-8e45-21ac605a7c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:04.729706 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79117bd7-520e-4780-8e45-21ac605a7c07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:04.742474 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79117bd7-520e-4780-8e45-21ac605a7c07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:04.744080 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79117bd7-520e-4780-8e45-21ac605a7c07 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:04.763711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-79117bd7-520e-4780-8e45-21ac605a7c07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:13:04.763711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:04.763711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:04.763711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:04.763711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:04.763711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:04.763711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:04.763711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:04.763711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:04.763711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:04.763711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:04.763711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:04.763711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:13:04.763711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:13:04.763711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:13:04.764968 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:13:04.764968 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:04.764968 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:04.764968 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:04.764968 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:04.764968 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:04.764968 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:04.764968 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:04.764968 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:13:04.764968 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:04.764968 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:04.764968 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:04.764968 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:04.764968 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:04.764968 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:04.764968 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:04.764968 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:04.766092 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:13:04.766092 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:13:04.766092 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:13:04.766092 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:13:04.766092 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 635/1251] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:13:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:13:04.773696 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab81abf-4762-4abc-834d-0b68dc82e4cf None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:04.885234 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab81abf-4762-4abc-834d-0b68dc82e4cf None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:04.926096 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff1f2cf-a554-437b-a8ad-afb70cbcca7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:04.926825 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff1f2cf-a554-437b-a8ad-afb70cbcca7e None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:04.927087 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff1f2cf-a554-437b-a8ad-afb70cbcca7e None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:04.927401 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff1f2cf-a554-437b-a8ad-afb70cbcca7e None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:04.928235 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff1f2cf-a554-437b-a8ad-afb70cbcca7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:04.928579 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff1f2cf-a554-437b-a8ad-afb70cbcca7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:04.930166 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff1f2cf-a554-437b-a8ad-afb70cbcca7e None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:04.944227 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 617/1252] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:13:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:04.950695 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79117bd7-520e-4780-8e45-21ac605a7c07 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:05.074273 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79117bd7-520e-4780-8e45-21ac605a7c07 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:05.105424 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:05.116098 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98502d-41f4-483d-ba43-90d6bf8544b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:05.117053 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98502d-41f4-483d-ba43-90d6bf8544b9 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:05.117336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98502d-41f4-483d-ba43-90d6bf8544b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:05.117636 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f98502d-41f4-483d-ba43-90d6bf8544b9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:05.127296 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f98502d-41f4-483d-ba43-90d6bf8544b9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:05.128966 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f98502d-41f4-483d-ba43-90d6bf8544b9 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:05.145640 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-6f98502d-41f4-483d-ba43-90d6bf8544b9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 03:13:05.145640 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:05.145640 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:05.145640 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:05.145640 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:05.145640 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:05.145640 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:05.145640 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:05.145640 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:05.145640 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:05.145640 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:05.145640 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:05.145640 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:13:05.145640 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:13:05.145640 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:13:05.145640 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:13:05.147057 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:05.147057 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:05.147057 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:13:05.147057 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:13:05.147057 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:13:05.147057 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:13:05.147057 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:05.147057 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:05.147057 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:05.147057 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:05.147057 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:05.147057 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:13:05.147057 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:05.147057 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:05.147057 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:05.147057 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:05.147057 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:05.147057 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:05.148278 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:05.148278 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:05.148278 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:13:05.148278 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:13:05.148278 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:13:05.148278 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:13:05.148278 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:13:05.148278 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:13:05.148278 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 03:13:05.148278 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:13:05.148278 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 636/1253] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:13:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:13:05.155832 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff1f2cf-a554-437b-a8ad-afb70cbcca7e None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:05.224600 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:05.280401 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff1f2cf-a554-437b-a8ad-afb70cbcca7e None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:05.322491 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cffca9-ea21-4493-8905-53b1b471494b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:05.323222 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cffca9-ea21-4493-8905-53b1b471494b None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:05.323488 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cffca9-ea21-4493-8905-53b1b471494b None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:05.323785 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cffca9-ea21-4493-8905-53b1b471494b None admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:05.324928 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cffca9-ea21-4493-8905-53b1b471494b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:05.325324 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cffca9-ea21-4493-8905-53b1b471494b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:05.327042 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cffca9-ea21-4493-8905-53b1b471494b None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:05.350896 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 618/1254] 10.222.0.50 () {56 vars in 1143 bytes} [Fri May 27 03:13:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:05.356881 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98502d-41f4-483d-ba43-90d6bf8544b9 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:05.470979 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f98502d-41f4-483d-ba43-90d6bf8544b9 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:05.504413 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1502a8-5bdf-4401-a946-90e464db9ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:05.505094 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1502a8-5bdf-4401-a946-90e464db9ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:05.505372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1502a8-5bdf-4401-a946-90e464db9ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:05.505642 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1502a8-5bdf-4401-a946-90e464db9ca1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:05.565401 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:05.580928 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1502a8-5bdf-4401-a946-90e464db9ca1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:05.582925 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1502a8-5bdf-4401-a946-90e464db9ca1 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:05.690634 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 637/1255] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:13:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:13:05.697920 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cffca9-ea21-4493-8905-53b1b471494b None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:05.805774 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cffca9-ea21-4493-8905-53b1b471494b None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:05.820348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:05.842026 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463d6205-e9c3-421e-bee6-3368884574a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:05.842855 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d6205-e9c3-421e-bee6-3368884574a1 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:05.843134 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d6205-e9c3-421e-bee6-3368884574a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:05.843406 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d6205-e9c3-421e-bee6-3368884574a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:05.853896 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463d6205-e9c3-421e-bee6-3368884574a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:05.855358 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463d6205-e9c3-421e-bee6-3368884574a1 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:05.873169 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-463d6205-e9c3-421e-bee6-3368884574a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:13:05.873169 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:05.873169 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:05.873169 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:05.873169 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:05.873169 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:05.873169 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:05.873169 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:05.873169 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:05.873169 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:05.873169 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:05.873169 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:05.873169 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:13:05.873169 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:13:05.873169 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:13:05.874277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:13:05.874277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:05.874277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:05.874277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:05.874277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:05.874277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:05.874277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:05.874277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:05.874277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:13:05.874277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:05.874277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:05.874277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:05.874277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:05.874277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:05.874277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:05.874277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:05.874277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:05.880499 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:13:05.880499 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:13:05.880499 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:13:05.880499 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:13:05.880499 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 619/1256] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:13:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:13:05.880842 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1502a8-5bdf-4401-a946-90e464db9ca1 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:05.974392 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:05.987459 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1502a8-5bdf-4401-a946-90e464db9ca1 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:06.027175 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686a8fd-02f9-4d7b-9296-1c1654499b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:06.028370 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686a8fd-02f9-4d7b-9296-1c1654499b54 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:06.028769 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686a8fd-02f9-4d7b-9296-1c1654499b54 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:06.029182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686a8fd-02f9-4d7b-9296-1c1654499b54 None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:06.030294 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d686a8fd-02f9-4d7b-9296-1c1654499b54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:06.030603 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d686a8fd-02f9-4d7b-9296-1c1654499b54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:06.032443 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d686a8fd-02f9-4d7b-9296-1c1654499b54 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:06.048267 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 638/1257] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:13:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:06.055583 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463d6205-e9c3-421e-bee6-3368884574a1 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:06.166787 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463d6205-e9c3-421e-bee6-3368884574a1 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:06.206340 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb9fe99-3628-42fc-af1b-b18286199923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:06.207149 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb9fe99-3628-42fc-af1b-b18286199923 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:06.207431 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb9fe99-3628-42fc-af1b-b18286199923 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:06.207725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb9fe99-3628-42fc-af1b-b18286199923 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:06.214885 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb9fe99-3628-42fc-af1b-b18286199923 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:06.216336 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb9fe99-3628-42fc-af1b-b18286199923 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:06.231624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-4bb9fe99-3628-42fc-af1b-b18286199923 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:13:06.231624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:06.231624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:06.231624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:06.231624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:06.231624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:06.231624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:06.231624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:06.231624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:06.231624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:06.231624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:06.231624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:06.231624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:13:06.231624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:13:06.231624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:13:06.232557 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:13:06.232557 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:06.232557 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:06.232557 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:06.232557 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:06.232557 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:06.232557 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:06.232557 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:06.232557 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:13:06.232557 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:06.232557 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:06.232557 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:06.232557 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:06.232557 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:06.232557 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:06.232557 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:06.232557 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:06.233474 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:13:06.233474 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:13:06.233474 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:13:06.233474 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:13:06.233474 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:13:06.233474 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:13:06.233474 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:13:06.233474 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:13:06.233474 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 620/1258] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:13:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:13:06.239081 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686a8fd-02f9-4d7b-9296-1c1654499b54 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:06.348069 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686a8fd-02f9-4d7b-9296-1c1654499b54 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:06.383152 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de86a0a-b514-43a0-b043-e5e47fecff0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:06.383822 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de86a0a-b514-43a0-b043-e5e47fecff0d None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:06.384096 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de86a0a-b514-43a0-b043-e5e47fecff0d None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:06.384390 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de86a0a-b514-43a0-b043-e5e47fecff0d None admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:06.385174 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de86a0a-b514-43a0-b043-e5e47fecff0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:06.385478 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de86a0a-b514-43a0-b043-e5e47fecff0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:06.386947 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de86a0a-b514-43a0-b043-e5e47fecff0d None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:06.399320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 639/1259] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 03:13:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:06.405787 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb9fe99-3628-42fc-af1b-b18286199923 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:06.432415 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:06.517238 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb9fe99-3628-42fc-af1b-b18286199923 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:06.556417 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52860446-b527-4e20-9275-4e7c93feb2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:06.557269 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52860446-b527-4e20-9275-4e7c93feb2db None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:06.557554 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52860446-b527-4e20-9275-4e7c93feb2db None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:06.557796 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52860446-b527-4e20-9275-4e7c93feb2db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:06.558634 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52860446-b527-4e20-9275-4e7c93feb2db None admin] RBAC: Adding query filter params (role.id=b4c524dd7bb44e94bd3b42d86e1cdf42, scope.project.id=f8398941252346468483ea0c863122f4, user.id=30963f2f464e4f54a8aea0175370a235) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:06.558982 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52860446-b527-4e20-9275-4e7c93feb2db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:06.560763 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52860446-b527-4e20-9275-4e7c93feb2db None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:06.567501 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 621/1260] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:13:06 2022] GET /identity/v3/role_assignments?user.id=30963f2f464e4f54a8aea0175370a235&scope.project.id=f8398941252346468483ea0c863122f4&role.id=b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:08.749180 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f23a8-bcb5-46a0-be0a-6f878eb64776 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:08.750069 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f23a8-bcb5-46a0-be0a-6f878eb64776 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:08.750069 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f23a8-bcb5-46a0-be0a-6f878eb64776 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:08.752247 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 640/1261] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:13:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:13:08.761295 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3b015-ff2c-4f57-bb07-fed6b63b4ba5 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:08.762003 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3b015-ff2c-4f57-bb07-fed6b63b4ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:08.762141 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3b015-ff2c-4f57-bb07-fed6b63b4ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:08.787741 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-31a3b015-ff2c-4f57-bb07-fed6b63b4ba5 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:13:08.788907 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-31a3b015-ff2c-4f57-bb07-fed6b63b4ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:13:08.863804 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 622/1262] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:13:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:13:08.886780 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f23a8-bcb5-46a0-be0a-6f878eb64776 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:08.904790 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:09.003230 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f23a8-bcb5-46a0-be0a-6f878eb64776 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:09.042331 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad529fd-9c59-4a5c-a30e-ccde77499464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:09.043232 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad529fd-9c59-4a5c-a30e-ccde77499464 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:09.043533 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad529fd-9c59-4a5c-a30e-ccde77499464 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:09.043856 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad529fd-9c59-4a5c-a30e-ccde77499464 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:09.055694 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad529fd-9c59-4a5c-a30e-ccde77499464 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:13:09.064320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-bad529fd-9c59-4a5c-a30e-ccde77499464 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:13:09.064320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:09.064320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:09.064320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:09.064320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:09.064320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:09.064320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:09.064320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:09.064320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:09.064320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:09.064320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:09.064320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:09.064320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:13:09.064320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:13:09.064320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:13:09.064320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:13:09.066262 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:13:09.066262 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:13:09.066262 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:13:09.066262 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:13:09.066262 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:09.066262 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:09.066262 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:09.066262 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:09.066262 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:09.066262 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:09.066262 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:09.066262 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:13:09.066262 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:09.066262 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:09.066262 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:09.066262 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:09.067470 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:09.067470 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:09.067470 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:09.067470 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:09.067470 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:13:09.067470 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:13:09.067470 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:13:09.067470 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:13:09.067470 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:13:09.067470 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:13:09.067470 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:13:09.067470 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:13:09.067470 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 641/1263] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 03:13:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:13:09.072070 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a3b015-ff2c-4f57-bb07-fed6b63b4ba5 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:09.090857 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:09.174372 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a3b015-ff2c-4f57-bb07-fed6b63b4ba5 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:09.213571 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3a4e6-5c67-4b42-901c-375fd34db871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:09.214233 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3a4e6-5c67-4b42-901c-375fd34db871 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:09.214516 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3a4e6-5c67-4b42-901c-375fd34db871 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:09.214794 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3a4e6-5c67-4b42-901c-375fd34db871 None admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:09.215545 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c3a4e6-5c67-4b42-901c-375fd34db871 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:09.215850 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c3a4e6-5c67-4b42-901c-375fd34db871 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:09.217254 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c3a4e6-5c67-4b42-901c-375fd34db871 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:09.225813 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 623/1264] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:13:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:09.232620 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad529fd-9c59-4a5c-a30e-ccde77499464 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:09.309614 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:09.346756 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad529fd-9c59-4a5c-a30e-ccde77499464 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:09.390904 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a433e-a613-49e7-989a-122aa90e1172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:09.391705 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a433e-a613-49e7-989a-122aa90e1172 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:09.391988 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a433e-a613-49e7-989a-122aa90e1172 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:09.392265 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a433e-a613-49e7-989a-122aa90e1172 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:09.403112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5a433e-a613-49e7-989a-122aa90e1172 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:09.404580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5a433e-a613-49e7-989a-122aa90e1172 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:09.422221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-de5a433e-a613-49e7-989a-122aa90e1172 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:13:09.422221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:09.422221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:09.422221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:09.422221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:09.422221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:09.422221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:09.422221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:09.422221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:09.422221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:09.422221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:09.422221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:09.422221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:13:09.422221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:13:09.422221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:13:09.423187 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:13:09.423187 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:09.423187 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:09.423187 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:09.423187 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:09.423187 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:09.423187 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:09.423187 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:09.423187 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:13:09.423187 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:09.423187 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:09.423187 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:09.423187 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:09.423187 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:09.423187 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:09.423187 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:09.423187 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:09.424334 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:13:09.424334 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:13:09.424334 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:13:09.424334 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:13:09.424334 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 642/1265] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:13:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:13:09.429622 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3a4e6-5c67-4b42-901c-375fd34db871 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:09.517080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:09.537764 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3a4e6-5c67-4b42-901c-375fd34db871 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:09.575337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3def4b-67e8-4356-a1fa-51c5c513f1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:09.575924 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3def4b-67e8-4356-a1fa-51c5c513f1e5 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:09.576205 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3def4b-67e8-4356-a1fa-51c5c513f1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:09.576478 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3def4b-67e8-4356-a1fa-51c5c513f1e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:09.577253 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3def4b-67e8-4356-a1fa-51c5c513f1e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:09.577556 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3def4b-67e8-4356-a1fa-51c5c513f1e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:09.578945 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3def4b-67e8-4356-a1fa-51c5c513f1e5 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:09.591626 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 624/1266] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:13:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:09.598514 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a433e-a613-49e7-989a-122aa90e1172 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:09.715773 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a433e-a613-49e7-989a-122aa90e1172 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:09.738156 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:09.754340 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61abd95-fe2f-421c-9b88-0e3a45180631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:09.755128 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61abd95-fe2f-421c-9b88-0e3a45180631 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:09.755401 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61abd95-fe2f-421c-9b88-0e3a45180631 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:09.755671 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61abd95-fe2f-421c-9b88-0e3a45180631 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:09.764142 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61abd95-fe2f-421c-9b88-0e3a45180631 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:09.765652 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61abd95-fe2f-421c-9b88-0e3a45180631 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:09.779771 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-e61abd95-fe2f-421c-9b88-0e3a45180631 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 03:13:09.779771 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:09.779771 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:09.779771 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:09.779771 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:09.779771 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:09.779771 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:09.779771 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:09.779771 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:09.779771 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:09.779771 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:09.779771 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:09.779771 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:13:09.779771 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:13:09.779771 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:13:09.779771 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:13:09.781215 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:09.781215 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:09.781215 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:13:09.781215 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:13:09.781215 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:13:09.781215 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:13:09.781215 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:09.781215 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:09.781215 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:09.781215 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:09.781215 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:09.781215 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:13:09.781215 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:09.781215 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:09.781215 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:09.781215 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:09.781215 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:09.781215 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:09.782582 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:09.782582 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:09.782582 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:13:09.782582 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:13:09.782582 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:13:09.782582 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:13:09.782582 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:13:09.782582 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:13:09.782582 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 03:13:09.782582 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:13:09.782582 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 643/1267] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:13:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:13:09.789289 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3def4b-67e8-4356-a1fa-51c5c513f1e5 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:09.899737 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3def4b-67e8-4356-a1fa-51c5c513f1e5 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:09.935216 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac554aed-c201-4214-823f-c08fcedbe6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:09.935880 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac554aed-c201-4214-823f-c08fcedbe6d3 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:09.936164 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac554aed-c201-4214-823f-c08fcedbe6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:09.936436 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac554aed-c201-4214-823f-c08fcedbe6d3 None admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:09.937293 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac554aed-c201-4214-823f-c08fcedbe6d3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:09.937600 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac554aed-c201-4214-823f-c08fcedbe6d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:09.939040 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac554aed-c201-4214-823f-c08fcedbe6d3 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:09.947803 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:09.956600 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 625/1268] 10.222.0.50 () {56 vars in 1143 bytes} [Fri May 27 03:13:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:09.962974 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61abd95-fe2f-421c-9b88-0e3a45180631 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:10.082397 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61abd95-fe2f-421c-9b88-0e3a45180631 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:10.119826 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4a703-347c-44a8-8e12-a16771736615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:10.121003 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4a703-347c-44a8-8e12-a16771736615 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:10.121361 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4a703-347c-44a8-8e12-a16771736615 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:10.121711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4a703-347c-44a8-8e12-a16771736615 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:10.140071 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4a703-347c-44a8-8e12-a16771736615 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:10.142361 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d4a703-347c-44a8-8e12-a16771736615 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:10.169366 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-b9d4a703-347c-44a8-8e12-a16771736615 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:13:10.169366 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:10.169366 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:10.169366 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:10.169366 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:10.169366 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:10.169366 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:10.169366 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:10.169366 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:10.169366 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:10.169366 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:10.169366 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:10.169366 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:13:10.169366 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:13:10.169366 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:13:10.170394 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:13:10.170394 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:10.170394 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:10.170394 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:10.170394 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:10.170394 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:10.170394 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:10.170394 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:10.170394 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:13:10.170394 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:10.170394 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:10.170394 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:10.170394 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:10.170394 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:10.170394 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:10.170394 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:10.170394 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:10.171830 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:13:10.171830 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:13:10.171830 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:13:10.171830 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:13:10.171830 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 644/1269] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:13:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:13:10.179718 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac554aed-c201-4214-823f-c08fcedbe6d3 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:10.300534 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac554aed-c201-4214-823f-c08fcedbe6d3 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:10.336094 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564eb25-8323-47b8-8c1c-2d68c98c83d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:10.336749 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564eb25-8323-47b8-8c1c-2d68c98c83d8 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:10.337051 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564eb25-8323-47b8-8c1c-2d68c98c83d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:10.337316 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564eb25-8323-47b8-8c1c-2d68c98c83d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:10.338027 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3564eb25-8323-47b8-8c1c-2d68c98c83d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:10.338320 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3564eb25-8323-47b8-8c1c-2d68c98c83d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:10.339689 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3564eb25-8323-47b8-8c1c-2d68c98c83d8 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:10.351074 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 626/1270] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:13:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:10.359560 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4a703-347c-44a8-8e12-a16771736615 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:10.393891 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:10.463523 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4a703-347c-44a8-8e12-a16771736615 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:10.499514 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb707bd1-93f5-4171-a99f-0e9338592c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:10.500293 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb707bd1-93f5-4171-a99f-0e9338592c4e None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:10.500564 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb707bd1-93f5-4171-a99f-0e9338592c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:10.500832 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb707bd1-93f5-4171-a99f-0e9338592c4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:10.508320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb707bd1-93f5-4171-a99f-0e9338592c4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:10.509776 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb707bd1-93f5-4171-a99f-0e9338592c4e None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:10.524112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-bb707bd1-93f5-4171-a99f-0e9338592c4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:13:10.524112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:10.524112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:10.524112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:10.524112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:10.524112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:10.524112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:10.524112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:10.524112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:10.524112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:10.524112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:10.524112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:10.524112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:13:10.524112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:13:10.524112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:13:10.525174 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:13:10.525174 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:10.525174 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:10.525174 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:10.525174 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:10.525174 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:10.525174 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:10.525174 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:10.525174 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:13:10.525174 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:10.525174 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:10.525174 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:10.525174 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:10.525174 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:10.525174 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:10.525174 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:10.525174 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:10.526292 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:13:10.526292 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:13:10.526292 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:13:10.526292 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:13:10.526292 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:13:10.526292 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:13:10.526292 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:13:10.526292 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:13:10.526292 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 645/1271] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:13:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 03:13:10.533887 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564eb25-8323-47b8-8c1c-2d68c98c83d8 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:10.605407 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:10.652998 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564eb25-8323-47b8-8c1c-2d68c98c83d8 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:10.697065 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8165808-b401-44aa-a790-567b2878358c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:10.697716 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8165808-b401-44aa-a790-567b2878358c None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:10.697989 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8165808-b401-44aa-a790-567b2878358c None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:10.698253 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8165808-b401-44aa-a790-567b2878358c None admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:10.699031 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8165808-b401-44aa-a790-567b2878358c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:10.699325 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8165808-b401-44aa-a790-567b2878358c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:10.700742 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8165808-b401-44aa-a790-567b2878358c None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:10.712830 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 627/1272] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 03:13:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:10.722058 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb707bd1-93f5-4171-a99f-0e9338592c4e None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:10.819157 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:10.831165 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb707bd1-93f5-4171-a99f-0e9338592c4e None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:10.867445 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ef497-b68b-466e-b286-731083fe9463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:10.868322 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ef497-b68b-466e-b286-731083fe9463 None admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:10.868624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ef497-b68b-466e-b286-731083fe9463 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:10.868892 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ef497-b68b-466e-b286-731083fe9463 None admin] PATH_INFO: `/v3/projects/f8398941252346468483ea0c863122f4/users/30963f2f464e4f54a8aea0175370a235/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:10.871061 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84ef497-b68b-466e-b286-731083fe9463 None admin] RBAC: Authorizing `identity:create_grant(project_id=f8398941252346468483ea0c863122f4, user_id=30963f2f464e4f54a8aea0175370a235, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:10.872493 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84ef497-b68b-466e-b286-731083fe9463 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:10.887180 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 646/1273] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 03:13:10 2022] PUT /identity/v3/projects/f8398941252346468483ea0c863122f4/users/30963f2f464e4f54a8aea0175370a235/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:13:13.055540 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57115bc-774c-4f04-9010-a0844a604a22 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:13.056258 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57115bc-774c-4f04-9010-a0844a604a22 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:13.056258 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57115bc-774c-4f04-9010-a0844a604a22 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:13.058496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 628/1274] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:13:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:13:13.068171 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4febbb-e582-4d48-b4d1-d27036af5e20 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:13.068616 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4febbb-e582-4d48-b4d1-d27036af5e20 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:13.068736 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4febbb-e582-4d48-b4d1-d27036af5e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:13.093578 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-6e4febbb-e582-4d48-b4d1-d27036af5e20 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:13:13.094905 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-6e4febbb-e582-4d48-b4d1-d27036af5e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:13:13.169362 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 647/1275] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:13:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:13:13.194005 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57115bc-774c-4f04-9010-a0844a604a22 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:13.312709 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57115bc-774c-4f04-9010-a0844a604a22 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:13.333005 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:13.353152 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f7cc1-a521-48c5-8406-752dbf48cc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:13.353955 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f7cc1-a521-48c5-8406-752dbf48cc2e None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:13.354236 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f7cc1-a521-48c5-8406-752dbf48cc2e None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:13.354510 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f7cc1-a521-48c5-8406-752dbf48cc2e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:13.365747 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f7cc1-a521-48c5-8406-752dbf48cc2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 03:13:13.373390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-6d2f7cc1-a521-48c5-8406-752dbf48cc2e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:13:13.373390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:13.373390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:13.373390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:13.373390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:13.373390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:13.373390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:13.373390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:13.373390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:13.373390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:13.373390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:13.373390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:13.373390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 03:13:13.373390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 03:13:13.373390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 03:13:13.373390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 03:13:13.374564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 03:13:13.374564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 03:13:13.374564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 03:13:13.374564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 03:13:13.374564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:13.374564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:13.374564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:13.374564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:13.374564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:13.374564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:13.374564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:13.374564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:13:13.374564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:13.374564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:13.374564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:13.374564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:13.375419 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:13.375419 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:13.375419 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:13.375419 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:13.375419 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 03:13:13.375419 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 03:13:13.375419 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 03:13:13.375419 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 03:13:13.375419 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 03:13:13.375419 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 03:13:13.375419 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 03:13:13.375419 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:13:13.375419 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 629/1276] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 03:13:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:13:13.383284 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4febbb-e582-4d48-b4d1-d27036af5e20 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:13.468044 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:13.488654 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4febbb-e582-4d48-b4d1-d27036af5e20 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:13.526086 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef0286-33e8-4f90-84af-3ac08d4f458d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:13.526748 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef0286-33e8-4f90-84af-3ac08d4f458d None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:13.527026 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef0286-33e8-4f90-84af-3ac08d4f458d None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:13.527299 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef0286-33e8-4f90-84af-3ac08d4f458d None admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:13.528046 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ef0286-33e8-4f90-84af-3ac08d4f458d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:13.528347 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ef0286-33e8-4f90-84af-3ac08d4f458d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:13.530103 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ef0286-33e8-4f90-84af-3ac08d4f458d None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:13.538541 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 648/1277] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 03:13:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:13.545216 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f7cc1-a521-48c5-8406-752dbf48cc2e None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:13.667970 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f7cc1-a521-48c5-8406-752dbf48cc2e None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:13.706680 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf9fe5-dd65-4e01-9ce9-7937e3b287a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:13.707467 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf9fe5-dd65-4e01-9ce9-7937e3b287a5 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:13.707737 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf9fe5-dd65-4e01-9ce9-7937e3b287a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:13.708001 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf9fe5-dd65-4e01-9ce9-7937e3b287a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:13.719234 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf9fe5-dd65-4e01-9ce9-7937e3b287a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:13.720693 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf9fe5-dd65-4e01-9ce9-7937e3b287a5 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:13.738333 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-1dbf9fe5-dd65-4e01-9ce9-7937e3b287a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:13:13.738333 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:13.738333 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:13.738333 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:13.738333 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:13.738333 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:13.738333 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:13.738333 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:13.738333 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:13.738333 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:13.738333 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:13.738333 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:13.738333 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:13:13.738333 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:13:13.738333 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:13:13.739356 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:13:13.739356 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:13.739356 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:13.739356 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:13.739356 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:13.739356 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:13.739356 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:13.739356 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:13.739356 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:13:13.739356 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:13.739356 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:13.739356 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:13.739356 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:13.739356 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:13.739356 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:13.739356 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:13.739356 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:13.740419 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:13:13.740419 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:13:13.740419 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:13:13.740419 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:13:13.740419 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 630/1278] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:13:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:13:13.745428 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef0286-33e8-4f90-84af-3ac08d4f458d None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:13.853381 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef0286-33e8-4f90-84af-3ac08d4f458d None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:13.892101 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346b9bb-12a3-4cc2-86b5-5303426ca5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:13.892828 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346b9bb-12a3-4cc2-86b5-5303426ca5f7 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:13.893155 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346b9bb-12a3-4cc2-86b5-5303426ca5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:13.893442 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346b9bb-12a3-4cc2-86b5-5303426ca5f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:13.894160 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2346b9bb-12a3-4cc2-86b5-5303426ca5f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:13.894461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2346b9bb-12a3-4cc2-86b5-5303426ca5f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:13.895838 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2346b9bb-12a3-4cc2-86b5-5303426ca5f7 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:13.909049 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 649/1279] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:13.915109 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf9fe5-dd65-4e01-9ce9-7937e3b287a5 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:13.950003 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:14.041094 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf9fe5-dd65-4e01-9ce9-7937e3b287a5 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:14.080125 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92fa799-d946-48b6-8cc8-d37f136842b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:14.080943 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92fa799-d946-48b6-8cc8-d37f136842b1 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:14.081249 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92fa799-d946-48b6-8cc8-d37f136842b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:14.081516 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92fa799-d946-48b6-8cc8-d37f136842b1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:14.089947 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92fa799-d946-48b6-8cc8-d37f136842b1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:14.091397 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92fa799-d946-48b6-8cc8-d37f136842b1 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:14.106297 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-a92fa799-d946-48b6-8cc8-d37f136842b1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 03:13:14.106297 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:14.106297 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:14.106297 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:14.106297 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:14.106297 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:14.106297 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:14.106297 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:14.106297 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:14.106297 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:14.106297 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:14.106297 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:14.106297 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 03:13:14.106297 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 03:13:14.106297 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 03:13:14.106297 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 03:13:14.107348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:14.107348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:14.107348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 03:13:14.107348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:13:14.107348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 03:13:14.107348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 03:13:14.107348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:14.107348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:14.107348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:14.107348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:14.107348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:14.107348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:13:14.107348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:14.107348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:14.107348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:14.107348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:14.107348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:14.107348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:14.108337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:14.108337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:14.108337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 03:13:14.108337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 03:13:14.108337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 03:13:14.108337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 03:13:14.108337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 03:13:14.108337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 03:13:14.108337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 03:13:14.108337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:13:14.108337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 631/1280] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:13:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:13:14.115458 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346b9bb-12a3-4cc2-86b5-5303426ca5f7 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:14.119527 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:14.226802 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346b9bb-12a3-4cc2-86b5-5303426ca5f7 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:14.261507 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b890b7a-f01a-4305-83f7-7b2832977d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:14.262163 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b890b7a-f01a-4305-83f7-7b2832977d3e None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:14.262436 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b890b7a-f01a-4305-83f7-7b2832977d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:14.262713 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b890b7a-f01a-4305-83f7-7b2832977d3e None admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:14.263527 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b890b7a-f01a-4305-83f7-7b2832977d3e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:14.263832 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b890b7a-f01a-4305-83f7-7b2832977d3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:14.265288 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b890b7a-f01a-4305-83f7-7b2832977d3e None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:14.283880 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 650/1281] 10.222.0.50 () {56 vars in 1143 bytes} [Fri May 27 03:13:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:13:14.292314 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92fa799-d946-48b6-8cc8-d37f136842b1 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:14.391143 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:14.402917 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92fa799-d946-48b6-8cc8-d37f136842b1 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:14.447507 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8923d0ed-abea-49b0-ab96-88aa59a440ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:14.448152 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8923d0ed-abea-49b0-ab96-88aa59a440ee None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:14.448433 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8923d0ed-abea-49b0-ab96-88aa59a440ee None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:14.448703 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8923d0ed-abea-49b0-ab96-88aa59a440ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:14.526442 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8923d0ed-abea-49b0-ab96-88aa59a440ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:14.527818 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8923d0ed-abea-49b0-ab96-88aa59a440ee None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:14.630054 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:14.646010 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 632/1282] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 03:13:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 03:13:14.654427 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b890b7a-f01a-4305-83f7-7b2832977d3e None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:14.722863 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:14.781060 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b890b7a-f01a-4305-83f7-7b2832977d3e None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:14.827011 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553cfacf-da06-418f-a988-0e797c27aded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:14.827806 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553cfacf-da06-418f-a988-0e797c27aded None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:14.828089 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553cfacf-da06-418f-a988-0e797c27aded None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:14.828361 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553cfacf-da06-418f-a988-0e797c27aded None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:14.840155 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553cfacf-da06-418f-a988-0e797c27aded None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:14.841671 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553cfacf-da06-418f-a988-0e797c27aded None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:14.864104 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-553cfacf-da06-418f-a988-0e797c27aded None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:13:14.864104 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:14.864104 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:14.864104 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:14.864104 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:14.864104 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:14.864104 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:14.864104 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:14.864104 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:14.864104 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:14.864104 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:14.864104 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:14.864104 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 03:13:14.864104 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 03:13:14.864104 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 03:13:14.865482 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 03:13:14.865482 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:14.865482 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:14.865482 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:14.865482 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:14.865482 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:14.865482 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:14.865482 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:14.865482 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:13:14.865482 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:14.865482 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:14.865482 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:14.865482 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:14.865482 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:14.865482 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:14.865482 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:14.865482 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:14.866364 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 03:13:14.866364 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 03:13:14.866364 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 03:13:14.866364 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:13:14.866364 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 651/1283] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:13:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:13:14.874373 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8923d0ed-abea-49b0-ab96-88aa59a440ee None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:14.985780 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8923d0ed-abea-49b0-ab96-88aa59a440ee None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:15.030580 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b0e76-c25d-4aa0-9671-f3dbb238c8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:15.031307 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b0e76-c25d-4aa0-9671-f3dbb238c8a0 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:15.031532 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b0e76-c25d-4aa0-9671-f3dbb238c8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:15.031799 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b0e76-c25d-4aa0-9671-f3dbb238c8a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:15.032539 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b0e76-c25d-4aa0-9671-f3dbb238c8a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:15.032842 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b0e76-c25d-4aa0-9671-f3dbb238c8a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:15.034890 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b0e76-c25d-4aa0-9671-f3dbb238c8a0 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:15.049028 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 633/1284] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 03:13:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:15.056214 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553cfacf-da06-418f-a988-0e797c27aded None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:15.179618 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553cfacf-da06-418f-a988-0e797c27aded None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:15.197889 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:15.223175 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251973b-b5a3-45c8-8450-39f0d9ec5a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:15.223971 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251973b-b5a3-45c8-8450-39f0d9ec5a22 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:15.224247 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251973b-b5a3-45c8-8450-39f0d9ec5a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:15.224525 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251973b-b5a3-45c8-8450-39f0d9ec5a22 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:15.232167 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b251973b-b5a3-45c8-8450-39f0d9ec5a22 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:15.233676 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b251973b-b5a3-45c8-8450-39f0d9ec5a22 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:15.248181 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application [None req-b251973b-b5a3-45c8-8450-39f0d9ec5a22 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:13:15.248181 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:15.248181 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:15.248181 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:15.248181 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:15.248181 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:15.248181 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:15.248181 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:15.248181 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:15.248181 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:15.248181 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:15.248181 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:15.248181 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 03:13:15.248181 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 03:13:15.248181 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 03:13:15.249309 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 03:13:15.249309 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:15.249309 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:15.249309 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:15.249309 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:15.249309 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:15.249309 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:15.249309 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:15.249309 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application with Lock( May 27 03:13:15.249309 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:15.249309 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:15.249309 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:15.249309 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:15.249309 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:15.249309 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:15.249309 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:15.249309 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:15.250612 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 03:13:15.250612 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 03:13:15.250612 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 03:13:15.250612 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 03:13:15.250612 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 03:13:15.250612 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 03:13:15.250612 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 03:13:15.250612 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: ERROR keystone.server.flask.application May 27 03:13:15.250612 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 652/1285] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 03:13:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:13:15.256828 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b0e76-c25d-4aa0-9671-f3dbb238c8a0 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:15.324302 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:15.388172 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b0e76-c25d-4aa0-9671-f3dbb238c8a0 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:15.434436 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67ff51d-2870-4dfc-9352-f8978f13a78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:15.435536 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ff51d-2870-4dfc-9352-f8978f13a78e None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:15.435889 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ff51d-2870-4dfc-9352-f8978f13a78e None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:15.436167 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67ff51d-2870-4dfc-9352-f8978f13a78e None admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:15.436982 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ff51d-2870-4dfc-9352-f8978f13a78e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:15.437429 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ff51d-2870-4dfc-9352-f8978f13a78e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:15.439608 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67ff51d-2870-4dfc-9352-f8978f13a78e None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:15.452896 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 634/1286] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 03:13:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:13:15.459429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251973b-b5a3-45c8-8450-39f0d9ec5a22 None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:15.564329 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251973b-b5a3-45c8-8450-39f0d9ec5a22 None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:15.599328 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3076b28-33ef-4dce-b45c-fed38cef4449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:15.599975 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3076b28-33ef-4dce-b45c-fed38cef4449 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:15.600251 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3076b28-33ef-4dce-b45c-fed38cef4449 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:15.600527 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3076b28-33ef-4dce-b45c-fed38cef4449 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:15.601363 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3076b28-33ef-4dce-b45c-fed38cef4449 None admin] RBAC: Adding query filter params (role.id=b4c524dd7bb44e94bd3b42d86e1cdf42, scope.project.id=f8398941252346468483ea0c863122f4, user.id=30963f2f464e4f54a8aea0175370a235) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:15.601675 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3076b28-33ef-4dce-b45c-fed38cef4449 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:15.603129 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3076b28-33ef-4dce-b45c-fed38cef4449 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:15.608939 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 653/1287] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 03:13:15 2022] GET /identity/v3/role_assignments?user.id=30963f2f464e4f54a8aea0175370a235&scope.project.id=f8398941252346468483ea0c863122f4&role.id=b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:17.794438 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f63b8-6225-4f9d-aeb8-25e2c2e859db None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:17.795035 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f63b8-6225-4f9d-aeb8-25e2c2e859db None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:17.795035 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f63b8-6225-4f9d-aeb8-25e2c2e859db None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:17.796371 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 635/1288] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:13:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:13:17.802824 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269ab98-649a-4c7e-8468-b7cd684499b5 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:17.803630 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269ab98-649a-4c7e-8468-b7cd684499b5 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:17.803980 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269ab98-649a-4c7e-8468-b7cd684499b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:17.824421 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-3269ab98-649a-4c7e-8468-b7cd684499b5 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:13:17.825662 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-3269ab98-649a-4c7e-8468-b7cd684499b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:13:17.858055 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:17.899548 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 654/1289] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:13:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:13:17.923169 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f63b8-6225-4f9d-aeb8-25e2c2e859db None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:18.040975 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f63b8-6225-4f9d-aeb8-25e2c2e859db None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:18.052172 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:18.078225 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a8105-2d23-40b7-9495-1b57500438f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:18.079036 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a8105-2d23-40b7-9495-1b57500438f3 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:18.079318 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a8105-2d23-40b7-9495-1b57500438f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:18.079594 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a8105-2d23-40b7-9495-1b57500438f3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:18.080254 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268a8105-2d23-40b7-9495-1b57500438f3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:18.081792 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268a8105-2d23-40b7-9495-1b57500438f3 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:18.098572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-268a8105-2d23-40b7-9495-1b57500438f3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 27 03:13:18.098572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:18.098572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:13:18.098572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:13:18.098572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:13:18.098572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:13:18.098572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:13:18.098572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:13:18.098572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 03:13:18.098572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:13:18.098572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:13:18.098572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:13:18.098572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:18.098572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:18.098572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:18.098572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:18.098572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:18.099766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:18.099766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:18.099766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:18.099766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:18.099766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:18.099766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:18.099766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:13:18.099766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:13:18.099766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:13:18.099766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:13:18.099766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:18.099766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:18.099766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:18.099766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:18.099766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:18.099766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:18.100811 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:18.100811 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:13:18.100811 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:18.100811 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:18.100811 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:18.100811 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:18.100811 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:18.100811 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:18.100811 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:18.100811 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:18.100811 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:13:18.100811 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:13:18.100811 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 03:13:18.100811 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:13:18.100811 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 636/1290] 10.222.0.50 () {56 vars in 1117 bytes} [Fri May 27 03:13:17 2022] GET /identity/v3/services/placement => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:13:18.106827 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3269ab98-649a-4c7e-8468-b7cd684499b5 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:18.216471 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3269ab98-649a-4c7e-8468-b7cd684499b5 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:18.253299 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ada5d-fd65-42fd-8ca5-ecb3f09fe466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:18.253973 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ada5d-fd65-42fd-8ca5-ecb3f09fe466 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:18.254255 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ada5d-fd65-42fd-8ca5-ecb3f09fe466 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:18.254524 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ada5d-fd65-42fd-8ca5-ecb3f09fe466 None admin] PATH_INFO: `/v3/services` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:18.255222 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ada5d-fd65-42fd-8ca5-ecb3f09fe466 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:18.255527 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ada5d-fd65-42fd-8ca5-ecb3f09fe466 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:18.256900 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ada5d-fd65-42fd-8ca5-ecb3f09fe466 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:18.261927 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 655/1291] 10.222.0.50 () {56 vars in 1116 bytes} [Fri May 27 03:13:18 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:18.270399 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a8105-2d23-40b7-9495-1b57500438f3 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:18.383260 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a8105-2d23-40b7-9495-1b57500438f3 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:18.424819 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7824d-b308-4122-bf9d-e04c86a93d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:18.426404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7824d-b308-4122-bf9d-e04c86a93d27 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:18.426800 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7824d-b308-4122-bf9d-e04c86a93d27 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:18.427158 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7824d-b308-4122-bf9d-e04c86a93d27 None admin] PATH_INFO: `/v3/services` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:18.428221 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7824d-b308-4122-bf9d-e04c86a93d27 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:18.428653 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7824d-b308-4122-bf9d-e04c86a93d27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:18.430793 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7824d-b308-4122-bf9d-e04c86a93d27 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:18.438962 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 637/1292] 10.222.0.50 () {56 vars in 1116 bytes} [Fri May 27 03:13:18 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:20.603504 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8003fa-67f0-4b2b-8c10-a58e07004d9e None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:20.604351 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8003fa-67f0-4b2b-8c10-a58e07004d9e None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:20.604351 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8003fa-67f0-4b2b-8c10-a58e07004d9e None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:20.606502 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 656/1293] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:13:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:13:20.613017 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da390454-7562-476c-94a3-9cadcb3cc605 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:20.613795 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da390454-7562-476c-94a3-9cadcb3cc605 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:20.614273 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da390454-7562-476c-94a3-9cadcb3cc605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:20.637082 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-da390454-7562-476c-94a3-9cadcb3cc605 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:13:20.638276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-da390454-7562-476c-94a3-9cadcb3cc605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:13:20.677695 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:20.716585 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 638/1294] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:13:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:13:20.739723 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8003fa-67f0-4b2b-8c10-a58e07004d9e None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:20.773748 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:20.850418 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8003fa-67f0-4b2b-8c10-a58e07004d9e None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:20.886016 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd72007-a455-4e99-9336-fb25498c5858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:20.886604 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd72007-a455-4e99-9336-fb25498c5858 None admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:20.886879 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd72007-a455-4e99-9336-fb25498c5858 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:20.887152 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd72007-a455-4e99-9336-fb25498c5858 None admin] PATH_INFO: `/v3/services` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:20.887863 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd72007-a455-4e99-9336-fb25498c5858 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:20.889273 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd72007-a455-4e99-9336-fb25498c5858 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:20.905537 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 657/1295] 10.222.0.50 () {60 vars in 1141 bytes} [Fri May 27 03:13:20 2022] POST /identity/v3/services => generated 246 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:13:23.142181 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987fcaa-1705-4f91-aafb-61c0adae43e6 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:23.142802 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987fcaa-1705-4f91-aafb-61c0adae43e6 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:23.142802 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8987fcaa-1705-4f91-aafb-61c0adae43e6 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:23.144206 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 639/1296] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:13:23.151234 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ce413-dd7a-4c55-bf4d-ad59da5c7bbc None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:23.151970 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ce413-dd7a-4c55-bf4d-ad59da5c7bbc None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:23.152129 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ce413-dd7a-4c55-bf4d-ad59da5c7bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:23.172219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-2f3ce413-dd7a-4c55-bf4d-ad59da5c7bbc None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:13:23.173451 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-2f3ce413-dd7a-4c55-bf4d-ad59da5c7bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:13:23.250727 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 658/1297] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:13:23 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:13:23.274228 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987fcaa-1705-4f91-aafb-61c0adae43e6 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:23.392760 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8987fcaa-1705-4f91-aafb-61c0adae43e6 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:23.429639 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c510c61c-c62b-4bed-8c29-31831d1ed365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:23.430449 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510c61c-c62b-4bed-8c29-31831d1ed365 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:23.430727 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510c61c-c62b-4bed-8c29-31831d1ed365 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:23.430998 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510c61c-c62b-4bed-8c29-31831d1ed365 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:23.431658 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c510c61c-c62b-4bed-8c29-31831d1ed365 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:23.433083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c510c61c-c62b-4bed-8c29-31831d1ed365 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:23.445001 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-c510c61c-c62b-4bed-8c29-31831d1ed365 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 27 03:13:23.445001 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:23.445001 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:13:23.445001 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:13:23.445001 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:13:23.445001 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:13:23.445001 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:13:23.445001 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:13:23.445001 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 03:13:23.445001 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:13:23.445001 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:13:23.445001 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:13:23.445001 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:23.445001 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:23.445001 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:23.445001 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:23.445001 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:23.446291 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:23.446291 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:23.446291 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:23.446291 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:23.446291 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:23.446291 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:23.446291 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:13:23.446291 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:13:23.446291 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:13:23.446291 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:13:23.446291 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:23.446291 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:23.446291 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:23.446291 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:23.446291 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:23.446291 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:23.447404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:23.447404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:13:23.447404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:23.447404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:23.447404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:23.447404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:23.447404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:23.447404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:23.447404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:23.447404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:23.447404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:13:23.447404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:13:23.447404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 03:13:23.447404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:13:23.447404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 640/1298] 10.222.0.50 () {56 vars in 1117 bytes} [Fri May 27 03:13:23 2022] GET /identity/v3/services/placement => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:13:23.452561 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ce413-dd7a-4c55-bf4d-ad59da5c7bbc None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:23.471264 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:23.563763 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ce413-dd7a-4c55-bf4d-ad59da5c7bbc None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:23.601410 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f80ca-e92c-44ff-b8a9-f9ae7505b98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:23.602065 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f80ca-e92c-44ff-b8a9-f9ae7505b98b None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:23.602343 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f80ca-e92c-44ff-b8a9-f9ae7505b98b None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:23.602616 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f80ca-e92c-44ff-b8a9-f9ae7505b98b None admin] PATH_INFO: `/v3/services` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:23.603322 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458f80ca-e92c-44ff-b8a9-f9ae7505b98b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:23.603628 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458f80ca-e92c-44ff-b8a9-f9ae7505b98b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:23.605011 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458f80ca-e92c-44ff-b8a9-f9ae7505b98b None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:23.610463 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 659/1299] 10.222.0.50 () {56 vars in 1116 bytes} [Fri May 27 03:13:23 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:23.616857 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c510c61c-c62b-4bed-8c29-31831d1ed365 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:23.653883 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:23.729708 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c510c61c-c62b-4bed-8c29-31831d1ed365 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:23.764789 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc73e94c-afac-42fd-b921-2b8a2d8bdee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:23.765462 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc73e94c-afac-42fd-b921-2b8a2d8bdee2 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:23.765745 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc73e94c-afac-42fd-b921-2b8a2d8bdee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:23.766019 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc73e94c-afac-42fd-b921-2b8a2d8bdee2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:23.766811 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc73e94c-afac-42fd-b921-2b8a2d8bdee2 None admin] RBAC: Adding query filter params (interface=public, service_id=a8010f68c09e4e73954d575998a9a3c2, region_id=RegionOne) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:23.767116 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc73e94c-afac-42fd-b921-2b8a2d8bdee2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:23.768498 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc73e94c-afac-42fd-b921-2b8a2d8bdee2 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:23.773774 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 641/1300] 10.222.0.50 () {56 vars in 1251 bytes} [Fri May 27 03:13:23 2022] GET /identity/v3/endpoints?service_id=a8010f68c09e4e73954d575998a9a3c2&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:13:23.780156 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f80ca-e92c-44ff-b8a9-f9ae7505b98b None admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:23.879385 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:23.911352 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f80ca-e92c-44ff-b8a9-f9ae7505b98b None admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:23.946861 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93dbb6f-0730-44d4-b66f-325176903fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:23.947531 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93dbb6f-0730-44d4-b66f-325176903fad None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:23.947807 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93dbb6f-0730-44d4-b66f-325176903fad None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:23.948080 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93dbb6f-0730-44d4-b66f-325176903fad None admin] PATH_INFO: `/v3/services` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:23.948759 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93dbb6f-0730-44d4-b66f-325176903fad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:23.950195 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93dbb6f-0730-44d4-b66f-325176903fad None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:23.957249 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 660/1301] 10.222.0.50 () {56 vars in 1087 bytes} [Fri May 27 03:13:23 2022] GET /identity/v3/services => generated 2191 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:13:26.143515 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02425d37-9bfa-4508-a95f-4f7fb4a8a9d9 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:26.144193 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02425d37-9bfa-4508-a95f-4f7fb4a8a9d9 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:26.144193 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02425d37-9bfa-4508-a95f-4f7fb4a8a9d9 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:26.145742 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 642/1302] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:13:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:13:26.152125 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73c1a3-61bb-4367-bffc-caa477e36210 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:26.152786 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73c1a3-61bb-4367-bffc-caa477e36210 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:26.153006 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73c1a3-61bb-4367-bffc-caa477e36210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:26.175386 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-6f73c1a3-61bb-4367-bffc-caa477e36210 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:13:26.176663 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-6f73c1a3-61bb-4367-bffc-caa477e36210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:13:26.248393 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 661/1303] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:13:26 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 03:13:26.280925 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02425d37-9bfa-4508-a95f-4f7fb4a8a9d9 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:26.385986 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:26.396831 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02425d37-9bfa-4508-a95f-4f7fb4a8a9d9 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:26.445029 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b89582-a808-463a-9090-17c776acf8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:26.445840 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b89582-a808-463a-9090-17c776acf8f0 None admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:26.446111 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b89582-a808-463a-9090-17c776acf8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:26.446371 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b89582-a808-463a-9090-17c776acf8f0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:26.447017 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b89582-a808-463a-9090-17c776acf8f0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:26.448382 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b89582-a808-463a-9090-17c776acf8f0 None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:26.461660 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application [None req-82b89582-a808-463a-9090-17c776acf8f0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 27 03:13:26.461660 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:26.461660 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 03:13:26.461660 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 03:13:26.461660 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 03:13:26.461660 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 03:13:26.461660 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 03:13:26.461660 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:13:26.461660 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 03:13:26.461660 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:13:26.461660 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 03:13:26.461660 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:13:26.461660 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 03:13:26.461660 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 03:13:26.461660 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 03:13:26.461660 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 03:13:26.461660 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 03:13:26.462829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 03:13:26.462829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 03:13:26.462829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 03:13:26.462829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 03:13:26.462829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 03:13:26.462829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 03:13:26.462829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 03:13:26.462829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 03:13:26.462829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 03:13:26.462829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 03:13:26.462829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 03:13:26.462829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 03:13:26.462829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 03:13:26.462829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 03:13:26.462829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 03:13:26.462829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.get_or_create( May 27 03:13:26.464011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 03:13:26.464011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application with Lock( May 27 03:13:26.464011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 03:13:26.464011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self._enter() May 27 03:13:26.464011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 03:13:26.464011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 03:13:26.464011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 03:13:26.464011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application return self.creator() May 27 03:13:26.464011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 03:13:26.464011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application created_value = creator( May 27 03:13:26.464011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 03:13:26.464011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 03:13:26.464011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 03:13:26.464011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: ERROR keystone.server.flask.application May 27 03:13:26.464011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 643/1304] 10.222.0.50 () {56 vars in 1117 bytes} [Fri May 27 03:13:26 2022] GET /identity/v3/services/placement => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 03:13:26.471868 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73c1a3-61bb-4367-bffc-caa477e36210 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:26.555552 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:26.579341 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73c1a3-61bb-4367-bffc-caa477e36210 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:26.614655 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf6de0-3174-423b-b9f0-1f1a310fae31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:26.615302 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf6de0-3174-423b-b9f0-1f1a310fae31 None admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:26.615571 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf6de0-3174-423b-b9f0-1f1a310fae31 None admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:26.615837 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf6de0-3174-423b-b9f0-1f1a310fae31 None admin] PATH_INFO: `/v3/services` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:26.616530 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf6de0-3174-423b-b9f0-1f1a310fae31 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:13:26.616827 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf6de0-3174-423b-b9f0-1f1a310fae31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:26.618227 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf6de0-3174-423b-b9f0-1f1a310fae31 None admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:26.623492 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 662/1305] 10.222.0.50 () {56 vars in 1116 bytes} [Fri May 27 03:13:26 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:13:26.629905 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b89582-a808-463a-9090-17c776acf8f0 None admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:13:26.743323 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b89582-a808-463a-9090-17c776acf8f0 None admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:13:26.781246 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bafe68-88f8-4078-8237-d4c2c6ebe6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:13:26.781892 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bafe68-88f8-4078-8237-d4c2c6ebe6aa None admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:13:26.782178 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bafe68-88f8-4078-8237-d4c2c6ebe6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:13:26.782437 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bafe68-88f8-4078-8237-d4c2c6ebe6aa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:13:26.783195 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bafe68-88f8-4078-8237-d4c2c6ebe6aa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:13:26.784641 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bafe68-88f8-4078-8237-d4c2c6ebe6aa None admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:13:26.807627 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 644/1306] 10.222.0.50 () {60 vars in 1144 bytes} [Fri May 27 03:13:26 2022] POST /identity/v3/endpoints => generated 331 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:13:36.620368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=30 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:13:36.803553 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:31.730589 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c662c-8150-44f1-8196-f0fb54eb898c None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:31.731030 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c662c-8150-44f1-8196-f0fb54eb898c None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:31.731565 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c662c-8150-44f1-8196-f0fb54eb898c None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:31.737585 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 663/1307] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:14:31 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:31.739185 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058cc72-dfd7-49ed-b657-95a7974f8ecf None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:31.739648 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058cc72-dfd7-49ed-b657-95a7974f8ecf None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:31.739732 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058cc72-dfd7-49ed-b657-95a7974f8ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:31.843154 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-6058cc72-dfd7-49ed-b657-95a7974f8ecf None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:31.846076 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-6058cc72-dfd7-49ed-b657-95a7974f8ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:32.026710 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 645/1308] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:14:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:14:32.083197 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6dd71e-23d9-47c7-b3d0-14ada057da33 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:32.083852 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6dd71e-23d9-47c7-b3d0-14ada057da33 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:32.084302 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6dd71e-23d9-47c7-b3d0-14ada057da33 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:32.086328 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 664/1309] 10.222.0.50 () {54 vars in 964 bytes} [Fri May 27 03:14:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:32.092828 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa28347-001b-4f24-a464-8098b7501b2e None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:32.093164 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa28347-001b-4f24-a464-8098b7501b2e None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:32.093438 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa28347-001b-4f24-a464-8098b7501b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:32.183139 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-cfa28347-001b-4f24-a464-8098b7501b2e None None] MFA Rules not processed for user `4ee30c0d6d694ee8945800157a9fc5b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:32.185354 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-cfa28347-001b-4f24-a464-8098b7501b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:32.220603 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:32.284455 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 646/1310] 10.222.0.50 () {58 vars in 1061 bytes} [Fri May 27 03:14:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:14:32.296330 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6dd71e-23d9-47c7-b3d0-14ada057da33 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:32.299387 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-3c6dd71e-23d9-47c7-b3d0-14ada057da33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:32.420805 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6dd71e-23d9-47c7-b3d0-14ada057da33 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:32.508404 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c13651-7bce-4519-a09c-1e011365222c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:32.509197 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c13651-7bce-4519-a09c-1e011365222c service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:32.509441 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c13651-7bce-4519-a09c-1e011365222c service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:32.509842 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c13651-7bce-4519-a09c-1e011365222c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:32.511701 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-d0c13651-7bce-4519-a09c-1e011365222c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:32.571733 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:32.630113 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c13651-7bce-4519-a09c-1e011365222c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:32.631668 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c13651-7bce-4519-a09c-1e011365222c service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:32.651592 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 665/1311] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:14:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:14:35.169226 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208d2812-714c-47ce-8cc3-3cb84dcde5ee None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:35.169779 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208d2812-714c-47ce-8cc3-3cb84dcde5ee None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:35.169779 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208d2812-714c-47ce-8cc3-3cb84dcde5ee None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:35.171311 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 647/1312] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:14:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:35.176871 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9304b5a-98e0-48ee-bc36-6a2b5a97f748 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:35.177428 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9304b5a-98e0-48ee-bc36-6a2b5a97f748 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:35.177428 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9304b5a-98e0-48ee-bc36-6a2b5a97f748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:35.260256 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-e9304b5a-98e0-48ee-bc36-6a2b5a97f748 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:35.261517 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-e9304b5a-98e0-48ee-bc36-6a2b5a97f748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:35.379724 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 666/1313] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:14:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:14:35.415960 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208d2812-714c-47ce-8cc3-3cb84dcde5ee None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:35.417666 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-208d2812-714c-47ce-8cc3-3cb84dcde5ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:35.530849 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208d2812-714c-47ce-8cc3-3cb84dcde5ee None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:35.605033 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:35.607724 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab72b75-c1ca-4529-86af-4ddeada0725a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:35.608384 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab72b75-c1ca-4529-86af-4ddeada0725a service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:35.608710 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab72b75-c1ca-4529-86af-4ddeada0725a service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:35.608992 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab72b75-c1ca-4529-86af-4ddeada0725a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:35.611208 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-cab72b75-c1ca-4529-86af-4ddeada0725a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:35.727756 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab72b75-c1ca-4529-86af-4ddeada0725a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:35.729235 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab72b75-c1ca-4529-86af-4ddeada0725a service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:35.749018 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 648/1314] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:14:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:14:38.268237 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08c00f-a62c-4927-bcef-9c824b07aaf9 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:38.268810 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08c00f-a62c-4927-bcef-9c824b07aaf9 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:38.268892 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b08c00f-a62c-4927-bcef-9c824b07aaf9 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:38.270370 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 667/1315] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:38.276997 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a3c2b-a8c8-4a41-81cc-164762f1f837 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:38.277432 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a3c2b-a8c8-4a41-81cc-164762f1f837 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:38.277584 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a3c2b-a8c8-4a41-81cc-164762f1f837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:38.348219 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-502a3c2b-a8c8-4a41-81cc-164762f1f837 None None] MFA Rules not processed for user `ba8ddcedde1a4bee9ba699929814f673`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:38.349455 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-502a3c2b-a8c8-4a41-81cc-164762f1f837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:38.403780 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:38.450360 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 649/1316] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:14:38 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:14:38.512567 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a6c7e-b9ff-424f-a86c-4eabd0e55298 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:38.512842 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a6c7e-b9ff-424f-a86c-4eabd0e55298 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:38.513274 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a6c7e-b9ff-424f-a86c-4eabd0e55298 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:38.515390 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 668/1317] 10.222.0.50 () {54 vars in 964 bytes} [Fri May 27 03:14:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:38.522623 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3022c71-2b64-4ab6-8794-29e4941dda45 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:38.522853 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3022c71-2b64-4ab6-8794-29e4941dda45 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:38.523138 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3022c71-2b64-4ab6-8794-29e4941dda45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:38.615846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-a3022c71-2b64-4ab6-8794-29e4941dda45 None None] MFA Rules not processed for user `4ee30c0d6d694ee8945800157a9fc5b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:38.617080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-a3022c71-2b64-4ab6-8794-29e4941dda45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:38.701211 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 650/1318] 10.222.0.50 () {58 vars in 1061 bytes} [Fri May 27 03:14:38 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:14:38.710652 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a6c7e-b9ff-424f-a86c-4eabd0e55298 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:38.712162 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-b45a6c7e-b9ff-424f-a86c-4eabd0e55298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:38.722242 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:38.824896 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a6c7e-b9ff-424f-a86c-4eabd0e55298 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:38.903875 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3bfc59-483d-45a4-863b-0af56062c5eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:38.906351 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3bfc59-483d-45a4-863b-0af56062c5eb service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:38.906638 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3bfc59-483d-45a4-863b-0af56062c5eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:38.906914 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3bfc59-483d-45a4-863b-0af56062c5eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:38.908622 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-2c3bfc59-483d-45a4-863b-0af56062c5eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:38.974905 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:39.027681 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3bfc59-483d-45a4-863b-0af56062c5eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:39.029540 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3bfc59-483d-45a4-863b-0af56062c5eb service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:39.049531 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 669/1319] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:14:38 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:14:42.234491 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ed086-233a-4fef-90cd-8f9e2fd1b014 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:42.235062 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ed086-233a-4fef-90cd-8f9e2fd1b014 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:42.235062 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ed086-233a-4fef-90cd-8f9e2fd1b014 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:42.236446 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 651/1320] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:14:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:42.242438 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbd7a2-9bde-45e3-bffb-a90d46c19b66 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:42.242864 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbd7a2-9bde-45e3-bffb-a90d46c19b66 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:42.242954 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbd7a2-9bde-45e3-bffb-a90d46c19b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:42.312912 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-9fbbd7a2-9bde-45e3-bffb-a90d46c19b66 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:42.314161 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-9fbbd7a2-9bde-45e3-bffb-a90d46c19b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:42.419704 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 670/1321] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:14:42 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:14:42.460275 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ed086-233a-4fef-90cd-8f9e2fd1b014 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:42.461835 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-b62ed086-233a-4fef-90cd-8f9e2fd1b014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:42.514800 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:42.572719 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ed086-233a-4fef-90cd-8f9e2fd1b014 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:42.653987 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe7fc00-6e5d-4295-84bb-f8b7fa7f42ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:42.654636 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe7fc00-6e5d-4295-84bb-f8b7fa7f42ec service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:42.654911 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe7fc00-6e5d-4295-84bb-f8b7fa7f42ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:42.655184 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe7fc00-6e5d-4295-84bb-f8b7fa7f42ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:42.656811 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-afe7fc00-6e5d-4295-84bb-f8b7fa7f42ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:42.761453 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:42.777470 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe7fc00-6e5d-4295-84bb-f8b7fa7f42ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:42.779190 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe7fc00-6e5d-4295-84bb-f8b7fa7f42ec service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:42.798174 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 652/1322] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:14:42 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:14:45.764702 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcab6c-dadf-48f9-84e3-42571fd7b454 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:45.765303 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcab6c-dadf-48f9-84e3-42571fd7b454 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:45.765485 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcab6c-dadf-48f9-84e3-42571fd7b454 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:45.766887 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 671/1323] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:14:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:45.772496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f703511-b561-4d15-a7bd-a4c8138a0292 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:45.773008 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f703511-b561-4d15-a7bd-a4c8138a0292 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:45.773106 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f703511-b561-4d15-a7bd-a4c8138a0292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:45.846576 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-6f703511-b561-4d15-a7bd-a4c8138a0292 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:45.847768 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-6f703511-b561-4d15-a7bd-a4c8138a0292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:45.943486 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 653/1324] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:14:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:14:45.979703 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bcab6c-dadf-48f9-84e3-42571fd7b454 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:45.981446 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-74bcab6c-dadf-48f9-84e3-42571fd7b454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:46.002342 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:46.095535 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bcab6c-dadf-48f9-84e3-42571fd7b454 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:46.184370 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b3acc-e053-4ab0-9173-1cf136c2f005 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:46.185051 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b3acc-e053-4ab0-9173-1cf136c2f005 service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:46.185336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b3acc-e053-4ab0-9173-1cf136c2f005 service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:46.185626 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b3acc-e053-4ab0-9173-1cf136c2f005 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:46.187262 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-588b3acc-e053-4ab0-9173-1cf136c2f005 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:46.279206 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:46.319179 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588b3acc-e053-4ab0-9173-1cf136c2f005 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:46.320690 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588b3acc-e053-4ab0-9173-1cf136c2f005 service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:46.342076 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 672/1325] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:14:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:14:48.601198 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce66e622-0727-473c-93ad-71a6511565c9 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:48.601763 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce66e622-0727-473c-93ad-71a6511565c9 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:48.601763 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce66e622-0727-473c-93ad-71a6511565c9 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:48.603022 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 654/1326] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:48.608576 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04410aa-8d75-4e3d-969c-1d184297754f None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:48.609007 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04410aa-8d75-4e3d-969c-1d184297754f None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:48.609368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04410aa-8d75-4e3d-969c-1d184297754f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:48.679740 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-b04410aa-8d75-4e3d-969c-1d184297754f None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:48.681108 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-b04410aa-8d75-4e3d-969c-1d184297754f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:48.780028 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 673/1327] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:14:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:14:48.816147 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce66e622-0727-473c-93ad-71a6511565c9 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:48.817778 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-ce66e622-0727-473c-93ad-71a6511565c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:48.881606 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:48.927333 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce66e622-0727-473c-93ad-71a6511565c9 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:49.008675 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f8612-50dd-42e1-afcd-1b161827d751 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:49.009366 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f8612-50dd-42e1-afcd-1b161827d751 service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:49.009628 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f8612-50dd-42e1-afcd-1b161827d751 service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:49.009923 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f8612-50dd-42e1-afcd-1b161827d751 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:49.011519 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-0e4f8612-50dd-42e1-afcd-1b161827d751 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:49.122933 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:49.126012 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4f8612-50dd-42e1-afcd-1b161827d751 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:49.128410 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4f8612-50dd-42e1-afcd-1b161827d751 service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:49.154133 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 655/1328] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:14:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:14:51.407073 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97def84a-f613-4e9c-9471-a7239c807e00 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:51.407621 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97def84a-f613-4e9c-9471-a7239c807e00 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:51.407621 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97def84a-f613-4e9c-9471-a7239c807e00 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:51.409195 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 674/1329] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:14:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:51.414691 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11d56c-0992-4032-ac25-c1ef82127651 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:51.415147 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11d56c-0992-4032-ac25-c1ef82127651 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:51.415235 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11d56c-0992-4032-ac25-c1ef82127651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:51.482044 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-cd11d56c-0992-4032-ac25-c1ef82127651 None None] MFA Rules not processed for user `ba8ddcedde1a4bee9ba699929814f673`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:51.483224 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-cd11d56c-0992-4032-ac25-c1ef82127651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:51.587466 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 656/1330] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:14:51 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:14:51.624913 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97def84a-f613-4e9c-9471-a7239c807e00 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:51.626556 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-97def84a-f613-4e9c-9471-a7239c807e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:51.661003 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:51.737273 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97def84a-f613-4e9c-9471-a7239c807e00 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:51.812237 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d0be58-cbf6-4b60-adef-e1ab1e7c7d82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:51.812820 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d0be58-cbf6-4b60-adef-e1ab1e7c7d82 service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:51.813108 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d0be58-cbf6-4b60-adef-e1ab1e7c7d82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:51.813377 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d0be58-cbf6-4b60-adef-e1ab1e7c7d82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:51.815662 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-d2d0be58-cbf6-4b60-adef-e1ab1e7c7d82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:51.895777 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:51.929769 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d0be58-cbf6-4b60-adef-e1ab1e7c7d82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:51.931228 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d0be58-cbf6-4b60-adef-e1ab1e7c7d82 service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:51.949691 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 675/1331] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:14:51 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:14:54.407478 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96a806-9ae5-40b6-a7d9-16cd79ad519a None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:54.408055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96a806-9ae5-40b6-a7d9-16cd79ad519a None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:54.408055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96a806-9ae5-40b6-a7d9-16cd79ad519a None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:54.409400 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 657/1332] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:14:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:54.415728 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f9655-6af2-4bbf-8c96-fe27612959e8 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:54.416491 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f9655-6af2-4bbf-8c96-fe27612959e8 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:54.416624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f9655-6af2-4bbf-8c96-fe27612959e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:54.487024 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-004f9655-6af2-4bbf-8c96-fe27612959e8 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:54.488176 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-004f9655-6af2-4bbf-8c96-fe27612959e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:54.595338 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 676/1333] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:14:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:14:54.630620 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96a806-9ae5-40b6-a7d9-16cd79ad519a None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:54.632154 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-ea96a806-9ae5-40b6-a7d9-16cd79ad519a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:54.714977 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:54.752605 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96a806-9ae5-40b6-a7d9-16cd79ad519a None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:54.835470 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e93441d-e39b-40f5-86f4-8e3a66b50468 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:54.836126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93441d-e39b-40f5-86f4-8e3a66b50468 service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:54.836421 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93441d-e39b-40f5-86f4-8e3a66b50468 service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:54.836671 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93441d-e39b-40f5-86f4-8e3a66b50468 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:54.838429 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-5e93441d-e39b-40f5-86f4-8e3a66b50468 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:54.962276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e93441d-e39b-40f5-86f4-8e3a66b50468 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:54.963988 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e93441d-e39b-40f5-86f4-8e3a66b50468 service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:54.975145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:54.984173 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 658/1334] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:14:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:14:57.233395 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5157e62-d20f-4614-ac44-340de8ae5353 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:57.233988 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5157e62-d20f-4614-ac44-340de8ae5353 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:57.233988 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5157e62-d20f-4614-ac44-340de8ae5353 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:57.236067 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 677/1335] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:14:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:14:57.244094 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330891e-8254-4596-b750-57f5e20c126e None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:57.244564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330891e-8254-4596-b750-57f5e20c126e None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:57.244659 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330891e-8254-4596-b750-57f5e20c126e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:57.315049 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-1330891e-8254-4596-b750-57f5e20c126e None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:14:57.316259 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-1330891e-8254-4596-b750-57f5e20c126e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:57.424924 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 659/1336] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:14:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:14:57.467428 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5157e62-d20f-4614-ac44-340de8ae5353 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:14:57.468980 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-f5157e62-d20f-4614-ac44-340de8ae5353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:57.513746 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:57.589465 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5157e62-d20f-4614-ac44-340de8ae5353 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:14:57.666757 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b21237-7a1c-4e44-8e62-8a3486aec57b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:14:57.667405 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b21237-7a1c-4e44-8e62-8a3486aec57b service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:14:57.667662 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b21237-7a1c-4e44-8e62-8a3486aec57b service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:14:57.667953 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b21237-7a1c-4e44-8e62-8a3486aec57b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:14:57.669610 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-47b21237-7a1c-4e44-8e62-8a3486aec57b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:14:57.760216 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:14:57.790146 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b21237-7a1c-4e44-8e62-8a3486aec57b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:14:57.791543 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b21237-7a1c-4e44-8e62-8a3486aec57b service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:14:57.810106 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 678/1337] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:14:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:15:00.453766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5146f-0b1c-41aa-9d04-5afdee3a8daa None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:00.454398 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5146f-0b1c-41aa-9d04-5afdee3a8daa None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:00.454398 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5146f-0b1c-41aa-9d04-5afdee3a8daa None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:00.456546 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 660/1338] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:15:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:15:00.463210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c8b83-5de5-4fba-a2a4-49480acaf5af None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:00.463906 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c8b83-5de5-4fba-a2a4-49480acaf5af None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:00.464098 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c8b83-5de5-4fba-a2a4-49480acaf5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:00.540433 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-432c8b83-5de5-4fba-a2a4-49480acaf5af None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:00.541658 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-432c8b83-5de5-4fba-a2a4-49480acaf5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:00.651246 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 679/1339] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:15:00 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:15:00.689235 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5146f-0b1c-41aa-9d04-5afdee3a8daa None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:00.690962 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-e1c5146f-0b1c-41aa-9d04-5afdee3a8daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:00.774164 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:00.803123 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5146f-0b1c-41aa-9d04-5afdee3a8daa None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:00.888677 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6aa509-d90c-4b56-a73f-600e2d708b5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:00.889371 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6aa509-d90c-4b56-a73f-600e2d708b5c service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:00.889649 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6aa509-d90c-4b56-a73f-600e2d708b5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:00.889924 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6aa509-d90c-4b56-a73f-600e2d708b5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:00.891582 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-2d6aa509-d90c-4b56-a73f-600e2d708b5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:01.019051 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6aa509-d90c-4b56-a73f-600e2d708b5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:01.020825 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6aa509-d90c-4b56-a73f-600e2d708b5c service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:01.043439 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 661/1340] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:15:00 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:15:05.300853 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528608f-06f2-48fd-9cdf-8d0ceeff5b2a None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:05.301373 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528608f-06f2-48fd-9cdf-8d0ceeff5b2a None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:05.301475 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8528608f-06f2-48fd-9cdf-8d0ceeff5b2a None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:05.302844 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 680/1341] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:15:05.308214 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c07f2-3820-4593-ad2a-85b7fb728e59 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:05.308623 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c07f2-3820-4593-ad2a-85b7fb728e59 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:05.308842 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5c07f2-3820-4593-ad2a-85b7fb728e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:05.315275 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:05.380219 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-4f5c07f2-3820-4593-ad2a-85b7fb728e59 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:05.381371 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-4f5c07f2-3820-4593-ad2a-85b7fb728e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:05.488893 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 662/1342] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:15:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:15:05.527194 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528608f-06f2-48fd-9cdf-8d0ceeff5b2a None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:05.530082 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-8528608f-06f2-48fd-9cdf-8d0ceeff5b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:05.584155 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:05.633819 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8528608f-06f2-48fd-9cdf-8d0ceeff5b2a None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:05.637555 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4704495-8237-4ecc-a595-597251a43b9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:05.638210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4704495-8237-4ecc-a595-597251a43b9f service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:05.638489 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4704495-8237-4ecc-a595-597251a43b9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:05.638759 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4704495-8237-4ecc-a595-597251a43b9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:05.648677 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4704495-8237-4ecc-a595-597251a43b9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:05.650185 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4704495-8237-4ecc-a595-597251a43b9f service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:05.660635 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 681/1343] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:15:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:15:08.405103 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5bcad-2568-4ccb-bc41-32e0dcedef6f None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:08.405663 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5bcad-2568-4ccb-bc41-32e0dcedef6f None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:08.405663 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5bcad-2568-4ccb-bc41-32e0dcedef6f None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:08.407026 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 663/1344] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:08.412509 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2329b17c-3f82-4c45-a1cd-0d5a19f860ba None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:08.412946 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2329b17c-3f82-4c45-a1cd-0d5a19f860ba None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:08.412946 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2329b17c-3f82-4c45-a1cd-0d5a19f860ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:08.441825 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-2329b17c-3f82-4c45-a1cd-0d5a19f860ba None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:08.443000 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-2329b17c-3f82-4c45-a1cd-0d5a19f860ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:08.525951 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 682/1345] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:15:08 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:15:08.563303 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d5bcad-2568-4ccb-bc41-32e0dcedef6f None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:08.574966 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d5bcad-2568-4ccb-bc41-32e0dcedef6f None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:08.578547 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1534dbcb-bf2c-4e42-9586-4abf65b04f5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:08.579259 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1534dbcb-bf2c-4e42-9586-4abf65b04f5b service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:08.580158 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1534dbcb-bf2c-4e42-9586-4abf65b04f5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:08.580563 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1534dbcb-bf2c-4e42-9586-4abf65b04f5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:08.590087 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1534dbcb-bf2c-4e42-9586-4abf65b04f5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:08.591541 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1534dbcb-bf2c-4e42-9586-4abf65b04f5b service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:08.601487 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 664/1346] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:15:08 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:15:12.812218 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69af7f-2681-4564-8e15-7fa91e62eaca None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:12.813119 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69af7f-2681-4564-8e15-7fa91e62eaca None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:12.813119 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69af7f-2681-4564-8e15-7fa91e62eaca None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:12.814785 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 683/1347] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:15:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:12.820635 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc46359-cf73-4aba-9789-f36a2fc65b9c None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:12.821214 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc46359-cf73-4aba-9789-f36a2fc65b9c None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:12.821214 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc46359-cf73-4aba-9789-f36a2fc65b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:12.843969 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-bcc46359-cf73-4aba-9789-f36a2fc65b9c None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:12.845327 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-bcc46359-cf73-4aba-9789-f36a2fc65b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:12.854878 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 665/1348] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:15:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:15:12.892939 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab69af7f-2681-4564-8e15-7fa91e62eaca None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:12.895136 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-ab69af7f-2681-4564-8e15-7fa91e62eaca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:12.910649 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab69af7f-2681-4564-8e15-7fa91e62eaca None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:12.914921 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8469ecc6-d46e-442a-8fb0-62d61c68b612 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:12.915569 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469ecc6-d46e-442a-8fb0-62d61c68b612 service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:12.915839 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469ecc6-d46e-442a-8fb0-62d61c68b612 service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:12.916093 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8469ecc6-d46e-442a-8fb0-62d61c68b612 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:12.926614 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469ecc6-d46e-442a-8fb0-62d61c68b612 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:12.928060 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8469ecc6-d46e-442a-8fb0-62d61c68b612 service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:12.938584 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 684/1349] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:15:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:15:18.437715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d7c6d-049b-4472-bb00-1546393fd3f4 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:18.438637 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d7c6d-049b-4472-bb00-1546393fd3f4 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:18.439255 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868d7c6d-049b-4472-bb00-1546393fd3f4 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:18.441275 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 666/1350] 10.222.0.50 () {54 vars in 921 bytes} [Fri May 27 03:15:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:18.446975 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4bf28e-e435-41fb-8fb3-41c9a3bffd7b None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:18.448109 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4bf28e-e435-41fb-8fb3-41c9a3bffd7b None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:18.448529 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4bf28e-e435-41fb-8fb3-41c9a3bffd7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:18.524444 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-1f4bf28e-e435-41fb-8fb3-41c9a3bffd7b None None] MFA Rules not processed for user `30963f2f464e4f54a8aea0175370a235`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:18.526003 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-1f4bf28e-e435-41fb-8fb3-41c9a3bffd7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:18.555157 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:18.639811 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 685/1351] 10.222.0.50 () {58 vars in 1018 bytes} [Fri May 27 03:15:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:15:22.801964 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e8746-9c01-4f7d-9e4d-a63b11470c12 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:22.802540 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e8746-9c01-4f7d-9e4d-a63b11470c12 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:22.802540 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e8746-9c01-4f7d-9e4d-a63b11470c12 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:22.804128 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 667/1352] 10.222.0.50 () {54 vars in 921 bytes} [Fri May 27 03:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:22.809202 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3fbb8-779c-4449-9a68-dc9284c68579 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:22.809697 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3fbb8-779c-4449-9a68-dc9284c68579 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:22.810081 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3fbb8-779c-4449-9a68-dc9284c68579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:22.830581 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-04a3fbb8-779c-4449-9a68-dc9284c68579 None None] MFA Rules not processed for user `30963f2f464e4f54a8aea0175370a235`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:22.831900 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-04a3fbb8-779c-4449-9a68-dc9284c68579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:22.836853 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=72 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:22.842284 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 686/1353] 10.222.0.50 () {58 vars in 1018 bytes} [Fri May 27 03:15:22 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:15:24.506404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb2b79-1ca5-4b74-8318-f1a8d17f304c None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:24.506970 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb2b79-1ca5-4b74-8318-f1a8d17f304c None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:24.506970 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb2b79-1ca5-4b74-8318-f1a8d17f304c None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:24.508515 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 668/1354] 10.222.0.50 () {54 vars in 921 bytes} [Fri May 27 03:15:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:24.513356 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d811e8-590f-4f5f-bdc7-2f64e7f69752 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:24.513772 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d811e8-590f-4f5f-bdc7-2f64e7f69752 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:24.514086 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d811e8-590f-4f5f-bdc7-2f64e7f69752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:24.534546 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-d6d811e8-590f-4f5f-bdc7-2f64e7f69752 None None] MFA Rules not processed for user `30963f2f464e4f54a8aea0175370a235`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:24.535814 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-d6d811e8-590f-4f5f-bdc7-2f64e7f69752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:24.545608 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 687/1355] 10.222.0.50 () {58 vars in 1018 bytes} [Fri May 27 03:15:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:15:28.443246 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2e0b0-380e-4bf2-b287-f0bc6937c95b None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:28.443885 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2e0b0-380e-4bf2-b287-f0bc6937c95b None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:28.443885 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2e0b0-380e-4bf2-b287-f0bc6937c95b None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:28.448985 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 669/1356] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:15:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:28.452499 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a96f172-7208-41d3-ba98-b61b752139f6 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:28.455257 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a96f172-7208-41d3-ba98-b61b752139f6 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:28.455660 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a96f172-7208-41d3-ba98-b61b752139f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:28.481177 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-8a96f172-7208-41d3-ba98-b61b752139f6 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:28.482666 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-8a96f172-7208-41d3-ba98-b61b752139f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:28.490619 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 688/1357] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:15:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:15:28.544798 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda99b4-dbe0-4b0a-861e-e94b611013af None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:28.545083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda99b4-dbe0-4b0a-861e-e94b611013af None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:28.545568 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda99b4-dbe0-4b0a-861e-e94b611013af None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:28.546923 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 670/1358] 10.222.0.50 () {54 vars in 961 bytes} [Fri May 27 03:15:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:28.556103 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311b3a0-1613-440a-ad3d-1e50a8ff5cc1 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:28.556637 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311b3a0-1613-440a-ad3d-1e50a8ff5cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:28.557170 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311b3a0-1613-440a-ad3d-1e50a8ff5cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:28.619152 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-7311b3a0-1613-440a-ad3d-1e50a8ff5cc1 None None] MFA Rules not processed for user `1a699988f08a4001b3b7d764f01f4fc5`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:28.620621 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-7311b3a0-1613-440a-ad3d-1e50a8ff5cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:28.725967 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 689/1359] 10.222.0.50 () {58 vars in 1058 bytes} [Fri May 27 03:15:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:15:28.733740 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda99b4-dbe0-4b0a-861e-e94b611013af None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:28.749504 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda99b4-dbe0-4b0a-861e-e94b611013af None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:28.755763 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc06b8b3-b943-4fc6-b68d-a8268ce5c988 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:28.756618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06b8b3-b943-4fc6-b68d-a8268ce5c988 service nova] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:28.757208 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06b8b3-b943-4fc6-b68d-a8268ce5c988 service nova] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:28.757693 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc06b8b3-b943-4fc6-b68d-a8268ce5c988 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:28.767332 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc06b8b3-b943-4fc6-b68d-a8268ce5c988 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:28.769168 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc06b8b3-b943-4fc6-b68d-a8268ce5c988 service nova] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:28.780465 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 671/1360] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:15:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:15:31.219200 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e82e44-7838-4bf6-993b-36f1747d7dc9 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:31.219782 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e82e44-7838-4bf6-993b-36f1747d7dc9 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:31.219782 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e82e44-7838-4bf6-993b-36f1747d7dc9 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:31.221254 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 690/1361] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:31.226658 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee88db-76ef-40de-a009-c337180faccf None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:31.227140 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee88db-76ef-40de-a009-c337180faccf None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:31.227140 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee88db-76ef-40de-a009-c337180faccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:31.245593 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-f8ee88db-76ef-40de-a009-c337180faccf None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:31.246852 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-f8ee88db-76ef-40de-a009-c337180faccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:31.255216 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 672/1362] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:15:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:15:31.299643 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e82e44-7838-4bf6-993b-36f1747d7dc9 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:31.314365 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e82e44-7838-4bf6-993b-36f1747d7dc9 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:31.319739 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d8422-15b4-46ac-9f6d-c6fe5b815cdf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:31.320383 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d8422-15b4-46ac-9f6d-c6fe5b815cdf service nova] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:31.320660 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d8422-15b4-46ac-9f6d-c6fe5b815cdf service nova] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:31.320933 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d8422-15b4-46ac-9f6d-c6fe5b815cdf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:31.330035 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75d8422-15b4-46ac-9f6d-c6fe5b815cdf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:31.331650 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75d8422-15b4-46ac-9f6d-c6fe5b815cdf service nova] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:31.341871 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 691/1363] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:15:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:15:33.926161 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cd6e8-c0d2-4357-9121-99b8655916b8 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:33.926765 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cd6e8-c0d2-4357-9121-99b8655916b8 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:33.926765 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cd6e8-c0d2-4357-9121-99b8655916b8 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:33.928443 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 673/1364] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:15:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:33.934644 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1923d6-905d-40dd-a135-498f4f318502 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:33.935484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1923d6-905d-40dd-a135-498f4f318502 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:33.936095 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1923d6-905d-40dd-a135-498f4f318502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:33.957384 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-7f1923d6-905d-40dd-a135-498f4f318502 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:33.958575 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-7f1923d6-905d-40dd-a135-498f4f318502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:33.966685 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 692/1365] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:15:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:15:34.024927 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26635e7-045e-4988-9a08-781581f20a77 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:34.025244 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26635e7-045e-4988-9a08-781581f20a77 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:34.025522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26635e7-045e-4988-9a08-781581f20a77 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:34.027299 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 674/1366] 10.222.0.50 () {54 vars in 961 bytes} [Fri May 27 03:15:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:34.034290 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8968e-a975-48ca-9918-66be8efbf190 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:34.034788 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8968e-a975-48ca-9918-66be8efbf190 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:34.035258 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8968e-a975-48ca-9918-66be8efbf190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:34.068854 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-edc8968e-a975-48ca-9918-66be8efbf190 None None] MFA Rules not processed for user `1a699988f08a4001b3b7d764f01f4fc5`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:34.070150 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-edc8968e-a975-48ca-9918-66be8efbf190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:34.082875 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 693/1367] 10.222.0.50 () {58 vars in 1058 bytes} [Fri May 27 03:15:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:15:34.091266 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26635e7-045e-4988-9a08-781581f20a77 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:34.106469 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26635e7-045e-4988-9a08-781581f20a77 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:34.111991 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da5d00-9c7c-4e6b-9fa8-9d21bd892f19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:34.112636 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da5d00-9c7c-4e6b-9fa8-9d21bd892f19 service nova] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:34.112912 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da5d00-9c7c-4e6b-9fa8-9d21bd892f19 service nova] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:34.113214 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da5d00-9c7c-4e6b-9fa8-9d21bd892f19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:34.121282 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da5d00-9c7c-4e6b-9fa8-9d21bd892f19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:34.122860 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da5d00-9c7c-4e6b-9fa8-9d21bd892f19 service nova] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:34.132054 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 675/1368] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:15:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:15:36.401433 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56281528-b2ec-4646-9784-7ca5155360bd None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:36.402669 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56281528-b2ec-4646-9784-7ca5155360bd None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:36.403095 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56281528-b2ec-4646-9784-7ca5155360bd None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:36.405007 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 694/1369] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:15:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:36.410825 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eecff8-5476-40e6-8963-635f4045ebfe None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:36.411250 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eecff8-5476-40e6-8963-635f4045ebfe None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:36.411384 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eecff8-5476-40e6-8963-635f4045ebfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:36.430626 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-16eecff8-5476-40e6-8963-635f4045ebfe None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:36.432130 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-16eecff8-5476-40e6-8963-635f4045ebfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:36.440706 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 676/1370] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:15:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:15:36.484546 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56281528-b2ec-4646-9784-7ca5155360bd None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:36.501404 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56281528-b2ec-4646-9784-7ca5155360bd None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:36.508819 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a221ab5-2498-4cc1-8f0a-0b769bd51a02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:36.509631 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a221ab5-2498-4cc1-8f0a-0b769bd51a02 service nova] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:36.510012 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a221ab5-2498-4cc1-8f0a-0b769bd51a02 service nova] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:36.510388 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a221ab5-2498-4cc1-8f0a-0b769bd51a02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:36.520755 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a221ab5-2498-4cc1-8f0a-0b769bd51a02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:36.522899 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a221ab5-2498-4cc1-8f0a-0b769bd51a02 service nova] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:36.536929 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 695/1371] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:15:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:15:36.729236 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60d1b7-875c-4afb-883d-79184b4b856b None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:36.729517 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60d1b7-875c-4afb-883d-79184b4b856b None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:36.729790 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60d1b7-875c-4afb-883d-79184b4b856b None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:36.731191 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 677/1372] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:15:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:36.737323 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1d188-06fe-4d44-a203-7513c233b3a1 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:36.737604 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1d188-06fe-4d44-a203-7513c233b3a1 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:36.738074 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d1d188-06fe-4d44-a203-7513c233b3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:36.757189 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-19d1d188-06fe-4d44-a203-7513c233b3a1 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:36.758427 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-19d1d188-06fe-4d44-a203-7513c233b3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:36.766973 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 696/1373] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:15:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:15:37.084046 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074d4e4-e02b-4aba-bc1e-64440300e0cf None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:37.084307 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074d4e4-e02b-4aba-bc1e-64440300e0cf None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:37.084571 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074d4e4-e02b-4aba-bc1e-64440300e0cf None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:37.086174 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 678/1374] 10.222.0.50 () {54 vars in 963 bytes} [Fri May 27 03:15:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:37.092223 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a2cb8-a0c1-4294-b63f-5b6a41f42f3c None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:37.092511 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a2cb8-a0c1-4294-b63f-5b6a41f42f3c None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:37.092787 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a2cb8-a0c1-4294-b63f-5b6a41f42f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:37.101418 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:37.150596 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-c71a2cb8-a0c1-4294-b63f-5b6a41f42f3c None None] MFA Rules not processed for user `36dd9b94ceb74eb4a7e9b5bbd5e8d8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:37.152238 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-c71a2cb8-a0c1-4294-b63f-5b6a41f42f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:37.217840 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 697/1375] 10.222.0.50 () {58 vars in 1060 bytes} [Fri May 27 03:15:37 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:15:37.225540 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e074d4e4-e02b-4aba-bc1e-64440300e0cf None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:37.237258 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e074d4e4-e02b-4aba-bc1e-64440300e0cf None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:37.241061 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a4bdeb-4019-40b3-8688-877ded84ba51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:37.241882 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a4bdeb-4019-40b3-8688-877ded84ba51 service cinder] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:37.242235 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a4bdeb-4019-40b3-8688-877ded84ba51 service cinder] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:37.242581 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a4bdeb-4019-40b3-8688-877ded84ba51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:37.252871 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a4bdeb-4019-40b3-8688-877ded84ba51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:37.254400 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a4bdeb-4019-40b3-8688-877ded84ba51 service cinder] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:37.265654 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 679/1376] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:15:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:15:38.801220 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9419d7-e4a9-480d-b836-81bd38ec0893 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:38.801851 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9419d7-e4a9-480d-b836-81bd38ec0893 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:38.801851 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9419d7-e4a9-480d-b836-81bd38ec0893 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:38.803162 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 698/1377] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:38.809252 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060361a-bbc0-4c0c-8ac3-8001d422639b None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:38.809809 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060361a-bbc0-4c0c-8ac3-8001d422639b None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:38.809809 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060361a-bbc0-4c0c-8ac3-8001d422639b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:38.829970 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-2060361a-bbc0-4c0c-8ac3-8001d422639b None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:38.831268 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-2060361a-bbc0-4c0c-8ac3-8001d422639b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:38.840147 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 680/1378] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:15:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:15:38.886677 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9419d7-e4a9-480d-b836-81bd38ec0893 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:38.902470 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9419d7-e4a9-480d-b836-81bd38ec0893 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:38.908683 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5b6b3-777a-4c0b-9489-658595107447 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:38.909379 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5b6b3-777a-4c0b-9489-658595107447 service nova] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:38.909655 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5b6b3-777a-4c0b-9489-658595107447 service nova] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:38.909923 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5b6b3-777a-4c0b-9489-658595107447 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:38.919277 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d5b6b3-777a-4c0b-9489-658595107447 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:38.920912 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d5b6b3-777a-4c0b-9489-658595107447 service nova] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:38.931924 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 699/1379] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:15:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:15:41.184882 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf4ec3-61b1-40d0-b4b5-aa797835273b None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:41.185791 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf4ec3-61b1-40d0-b4b5-aa797835273b None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:41.186221 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf4ec3-61b1-40d0-b4b5-aa797835273b None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:41.189436 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 681/1380] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:15:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:41.194010 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0c724-452c-4a00-b0b8-19cc5bc8921c None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:41.194463 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0c724-452c-4a00-b0b8-19cc5bc8921c None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:41.194743 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0c724-452c-4a00-b0b8-19cc5bc8921c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:41.214659 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-12c0c724-452c-4a00-b0b8-19cc5bc8921c None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:41.216058 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-12c0c724-452c-4a00-b0b8-19cc5bc8921c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:41.225758 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 700/1381] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:15:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:15:41.275780 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf4ec3-61b1-40d0-b4b5-aa797835273b None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:41.292229 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf4ec3-61b1-40d0-b4b5-aa797835273b None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:41.298208 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c1420-9267-4055-a325-f31f8af2c501 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:41.299130 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c1420-9267-4055-a325-f31f8af2c501 service nova] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:41.300100 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c1420-9267-4055-a325-f31f8af2c501 service nova] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:41.300549 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c1420-9267-4055-a325-f31f8af2c501 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:41.309932 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c1420-9267-4055-a325-f31f8af2c501 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:41.311698 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c1420-9267-4055-a325-f31f8af2c501 service nova] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:41.322181 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 682/1382] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:15:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:15:43.609457 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e562c-dd64-4152-a17c-17cb5ee20a73 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:43.610029 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e562c-dd64-4152-a17c-17cb5ee20a73 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:43.610029 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e562c-dd64-4152-a17c-17cb5ee20a73 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:43.611374 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 701/1383] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:15:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:43.617064 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952e772-2d25-48cb-9bab-2e0254dfea1f None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:43.617511 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952e772-2d25-48cb-9bab-2e0254dfea1f None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:43.617590 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952e772-2d25-48cb-9bab-2e0254dfea1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:43.636043 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-6952e772-2d25-48cb-9bab-2e0254dfea1f None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:43.637261 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-6952e772-2d25-48cb-9bab-2e0254dfea1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:43.645682 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 683/1384] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:15:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:15:43.687817 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e562c-dd64-4152-a17c-17cb5ee20a73 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:43.703876 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e562c-dd64-4152-a17c-17cb5ee20a73 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:43.709747 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1930dfc1-82c7-4390-a67d-41bc08081c37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:43.710432 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930dfc1-82c7-4390-a67d-41bc08081c37 service nova] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:43.710714 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930dfc1-82c7-4390-a67d-41bc08081c37 service nova] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:43.710980 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1930dfc1-82c7-4390-a67d-41bc08081c37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:43.721616 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1930dfc1-82c7-4390-a67d-41bc08081c37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:43.723540 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1930dfc1-82c7-4390-a67d-41bc08081c37 service nova] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:43.735788 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 702/1385] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:15:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:15:46.025659 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e67ebf-293a-424f-9e3f-d354bb8ea3c6 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:46.026214 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e67ebf-293a-424f-9e3f-d354bb8ea3c6 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:46.026671 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e67ebf-293a-424f-9e3f-d354bb8ea3c6 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:46.028037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 684/1386] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:15:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:46.033713 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dce9a3-1236-4a50-8b08-a396ae5da196 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:46.034160 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dce9a3-1236-4a50-8b08-a396ae5da196 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:46.034311 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dce9a3-1236-4a50-8b08-a396ae5da196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:46.053968 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-77dce9a3-1236-4a50-8b08-a396ae5da196 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:46.055240 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-77dce9a3-1236-4a50-8b08-a396ae5da196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:46.065048 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 703/1387] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:15:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:15:46.117383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e67ebf-293a-424f-9e3f-d354bb8ea3c6 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:46.136328 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e67ebf-293a-424f-9e3f-d354bb8ea3c6 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:46.143426 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d458fe8-3188-4da2-955f-cfb55e60a4d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:46.144234 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d458fe8-3188-4da2-955f-cfb55e60a4d1 service nova] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:46.144587 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d458fe8-3188-4da2-955f-cfb55e60a4d1 service nova] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:46.145227 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d458fe8-3188-4da2-955f-cfb55e60a4d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:46.156204 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d458fe8-3188-4da2-955f-cfb55e60a4d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:46.158251 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d458fe8-3188-4da2-955f-cfb55e60a4d1 service nova] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:46.170427 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 685/1388] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:15:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:15:48.772468 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cc163-850f-4fa9-8f01-4c622ee3521e None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:48.773127 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cc163-850f-4fa9-8f01-4c622ee3521e None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:48.773127 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cc163-850f-4fa9-8f01-4c622ee3521e None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:48.774516 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 704/1389] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:15:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:48.781753 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9530f-162f-478c-8f03-4ab2caee7575 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:48.782289 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9530f-162f-478c-8f03-4ab2caee7575 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:48.782431 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9530f-162f-478c-8f03-4ab2caee7575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:48.804596 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-19d9530f-162f-478c-8f03-4ab2caee7575 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:48.806112 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-19d9530f-162f-478c-8f03-4ab2caee7575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:48.816575 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 686/1390] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:15:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:15:48.870952 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6cc163-850f-4fa9-8f01-4c622ee3521e None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:48.888016 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6cc163-850f-4fa9-8f01-4c622ee3521e None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:48.894702 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9304978-f172-413b-a4bb-172363cb8fd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:48.895403 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9304978-f172-413b-a4bb-172363cb8fd9 service nova] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:48.895680 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9304978-f172-413b-a4bb-172363cb8fd9 service nova] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:48.895951 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9304978-f172-413b-a4bb-172363cb8fd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:48.905631 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9304978-f172-413b-a4bb-172363cb8fd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:48.908009 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9304978-f172-413b-a4bb-172363cb8fd9 service nova] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:48.917747 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 705/1391] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:15:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:15:51.472526 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1be2d-9a05-467a-ace2-5744813afb0a None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:51.473214 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1be2d-9a05-467a-ace2-5744813afb0a None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:51.473214 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1be2d-9a05-467a-ace2-5744813afb0a None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:51.474512 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 687/1392] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:15:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:51.479967 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaffb7df-200b-444e-bf95-c337a0aa24b6 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:51.480441 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaffb7df-200b-444e-bf95-c337a0aa24b6 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:51.480688 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaffb7df-200b-444e-bf95-c337a0aa24b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:51.499779 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-eaffb7df-200b-444e-bf95-c337a0aa24b6 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:51.501001 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-eaffb7df-200b-444e-bf95-c337a0aa24b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:51.509916 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 706/1393] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:15:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:15:51.554582 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb1be2d-9a05-467a-ace2-5744813afb0a None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:51.569520 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb1be2d-9a05-467a-ace2-5744813afb0a None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:51.575284 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f2d11f-45e8-4b01-a00b-a608374f8373 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:51.575923 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f2d11f-45e8-4b01-a00b-a608374f8373 service nova] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:51.576205 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f2d11f-45e8-4b01-a00b-a608374f8373 service nova] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:51.576477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f2d11f-45e8-4b01-a00b-a608374f8373 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:51.584841 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f2d11f-45e8-4b01-a00b-a608374f8373 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:51.586473 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f2d11f-45e8-4b01-a00b-a608374f8373 service nova] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:51.596061 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 688/1394] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:15:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:15:52.880469 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4da80-d78c-4616-8292-3f588d2f2b8b None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:52.881111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4da80-d78c-4616-8292-3f588d2f2b8b None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:52.881111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4da80-d78c-4616-8292-3f588d2f2b8b None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:52.882479 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 707/1395] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:52.888058 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec1a949-1dd3-4d9b-9b8e-4a23aa7fc949 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:52.888515 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec1a949-1dd3-4d9b-9b8e-4a23aa7fc949 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:52.888653 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec1a949-1dd3-4d9b-9b8e-4a23aa7fc949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:52.895340 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=52 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:52.909413 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-dec1a949-1dd3-4d9b-9b8e-4a23aa7fc949 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:52.910613 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-dec1a949-1dd3-4d9b-9b8e-4a23aa7fc949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:52.919025 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 689/1396] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:15:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:15:52.975216 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65726246-01bb-46d8-b25d-879598499c50 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:52.975508 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65726246-01bb-46d8-b25d-879598499c50 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:52.975786 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65726246-01bb-46d8-b25d-879598499c50 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:52.977307 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 708/1397] 10.222.0.50 () {54 vars in 962 bytes} [Fri May 27 03:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:52.982532 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610216c9-2876-424a-bff8-ab3f9cc7bc5f None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:52.982986 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610216c9-2876-424a-bff8-ab3f9cc7bc5f None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:52.983330 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610216c9-2876-424a-bff8-ab3f9cc7bc5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:53.034315 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-610216c9-2876-424a-bff8-ab3f9cc7bc5f None None] MFA Rules not processed for user `5b3c9c5c8d2e4376b1b70d974d4cb851`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:53.035529 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-610216c9-2876-424a-bff8-ab3f9cc7bc5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:53.096343 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 690/1398] 10.222.0.50 () {58 vars in 1059 bytes} [Fri May 27 03:15:52 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:15:53.104007 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65726246-01bb-46d8-b25d-879598499c50 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:53.115003 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65726246-01bb-46d8-b25d-879598499c50 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:53.118430 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd813119-29f2-4dbd-8b64-c3da5e845cc8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3c9c5c8d2e4376b1b70d974d4cb851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:53.119075 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd813119-29f2-4dbd-8b64-c3da5e845cc8 service glance] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:53.119357 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd813119-29f2-4dbd-8b64-c3da5e845cc8 service glance] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:53.119630 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd813119-29f2-4dbd-8b64-c3da5e845cc8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:53.128892 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd813119-29f2-4dbd-8b64-c3da5e845cc8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:53.130363 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd813119-29f2-4dbd-8b64-c3da5e845cc8 service glance] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:53.140341 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 709/1399] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:15:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:15:53.684789 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fddda1-e842-4e81-9acb-317817e63e75 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:53.685131 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fddda1-e842-4e81-9acb-317817e63e75 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:53.685434 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fddda1-e842-4e81-9acb-317817e63e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:53.734296 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-12fddda1-e842-4e81-9acb-317817e63e75 None None] MFA Rules not processed for user `86a18e313e4944748aef58d54bfe7b13`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:53.735564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-12fddda1-e842-4e81-9acb-317817e63e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:53.804808 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 691/1400] 10.222.0.50 () {58 vars in 1009 bytes} [Fri May 27 03:15:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:15:53.830466 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe042cb-cc69-4160-a9e0-c3f46335ea13 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:53.830749 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe042cb-cc69-4160-a9e0-c3f46335ea13 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:53.831026 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe042cb-cc69-4160-a9e0-c3f46335ea13 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:53.832422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 710/1401] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:53.838419 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e95bb6-d129-41b0-92bd-dc4cf7d2c02b None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:53.838624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e95bb6-d129-41b0-92bd-dc4cf7d2c02b None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:53.838808 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e95bb6-d129-41b0-92bd-dc4cf7d2c02b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:53.850136 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6b9fc-6940-4d32-9fe0-35ec05b626f9 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:53.850609 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6b9fc-6940-4d32-9fe0-35ec05b626f9 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:53.850885 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f6b9fc-6940-4d32-9fe0-35ec05b626f9 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:53.852525 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 711/1402] 10.222.0.50 () {54 vars in 942 bytes} [Fri May 27 03:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:53.858047 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-f4e95bb6-d129-41b0-92bd-dc4cf7d2c02b None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:53.859268 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac1fc4-4e4e-4e3b-9b63-4135417b759f None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:53.859475 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-f4e95bb6-d129-41b0-92bd-dc4cf7d2c02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:53.859592 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac1fc4-4e4e-4e3b-9b63-4135417b759f None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:53.859993 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac1fc4-4e4e-4e3b-9b63-4135417b759f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:53.868107 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 692/1403] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:15:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:15:53.909124 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-e7ac1fc4-4e4e-4e3b-9b63-4135417b759f None None] MFA Rules not processed for user `3de53e2f0fba4e0fac70887067c1963d`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:53.910408 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-e7ac1fc4-4e4e-4e3b-9b63-4135417b759f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:53.913765 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e95bb6-d129-41b0-92bd-dc4cf7d2c02b None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:53.926071 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e95bb6-d129-41b0-92bd-dc4cf7d2c02b None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:53.932088 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c7f5b1-3481-4f00-8f8f-d4180438c248 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:53.932740 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7f5b1-3481-4f00-8f8f-d4180438c248 service nova] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:53.933046 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7f5b1-3481-4f00-8f8f-d4180438c248 service nova] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:53.933461 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7f5b1-3481-4f00-8f8f-d4180438c248 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:53.941938 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c7f5b1-3481-4f00-8f8f-d4180438c248 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:53.943547 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c7f5b1-3481-4f00-8f8f-d4180438c248 service nova] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:53.951198 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:15:53.953184 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 693/1404] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:15:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:15:53.986851 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 712/1405] 10.222.0.50 () {58 vars in 1039 bytes} [Fri May 27 03:15:53 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:15:53.996327 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c7f5b1-3481-4f00-8f8f-d4180438c248 service nova] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:54.007710 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c7f5b1-3481-4f00-8f8f-d4180438c248 service nova] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:54.013449 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da940a1-cf77-4a1e-94ac-e529f3e290bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de53e2f0fba4e0fac70887067c1963d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:54.014151 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da940a1-cf77-4a1e-94ac-e529f3e290bc service swift] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:54.014435 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da940a1-cf77-4a1e-94ac-e529f3e290bc service swift] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:54.014705 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da940a1-cf77-4a1e-94ac-e529f3e290bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:54.030993 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da940a1-cf77-4a1e-94ac-e529f3e290bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:54.032746 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da940a1-cf77-4a1e-94ac-e529f3e290bc service swift] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:54.042733 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 694/1406] 10.222.0.50 () {60 vars in 1384 bytes} [Fri May 27 03:15:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:15:56.306457 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0687cd23-83d5-4992-9b26-76a2b4973f08 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:56.306960 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0687cd23-83d5-4992-9b26-76a2b4973f08 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:56.307044 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0687cd23-83d5-4992-9b26-76a2b4973f08 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:56.308573 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 713/1407] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:15:56.314747 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3126775d-8aab-4c9c-b9af-fa45c8abd5c7 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:56.315188 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3126775d-8aab-4c9c-b9af-fa45c8abd5c7 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:56.315288 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3126775d-8aab-4c9c-b9af-fa45c8abd5c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:56.333716 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-3126775d-8aab-4c9c-b9af-fa45c8abd5c7 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:15:56.335203 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-3126775d-8aab-4c9c-b9af-fa45c8abd5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:15:56.345015 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 695/1408] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:15:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:15:56.391671 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0687cd23-83d5-4992-9b26-76a2b4973f08 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:15:56.407632 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0687cd23-83d5-4992-9b26-76a2b4973f08 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:15:56.416759 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff7acd-376e-41b0-9c24-c3b075d9ba1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:15:56.417455 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff7acd-376e-41b0-9c24-c3b075d9ba1b service nova] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:15:56.417736 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff7acd-376e-41b0-9c24-c3b075d9ba1b service nova] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:15:56.418016 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff7acd-376e-41b0-9c24-c3b075d9ba1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:15:56.427818 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fff7acd-376e-41b0-9c24-c3b075d9ba1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:15:56.429473 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fff7acd-376e-41b0-9c24-c3b075d9ba1b service nova] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:15:56.442158 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 714/1409] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:15:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:16:06.571131 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=98 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:06.592749 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=19 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:17.843309 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e42a8-3b55-40a6-96cf-d625bae74c15 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:17.843897 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e42a8-3b55-40a6-96cf-d625bae74c15 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:17.843897 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e42a8-3b55-40a6-96cf-d625bae74c15 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:17.845304 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 696/1410] 10.222.0.50 () {54 vars in 918 bytes} [Fri May 27 03:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:17.850581 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc45c0-fee8-40d5-aae6-6a1e099cf2ac None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:17.851126 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc45c0-fee8-40d5-aae6-6a1e099cf2ac None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:17.851126 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc45c0-fee8-40d5-aae6-6a1e099cf2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:17.871249 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-d5dc45c0-fee8-40d5-aae6-6a1e099cf2ac None None] MFA Rules not processed for user `30963f2f464e4f54a8aea0175370a235`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:17.872448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-d5dc45c0-fee8-40d5-aae6-6a1e099cf2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:17.882402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 715/1411] 10.222.0.50 () {58 vars in 1015 bytes} [Fri May 27 03:16:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:16:17.896826 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff3f300-6ac9-4deb-9573-9e0dc08b6d50 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:17.897140 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff3f300-6ac9-4deb-9573-9e0dc08b6d50 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:17.897437 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff3f300-6ac9-4deb-9573-9e0dc08b6d50 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:17.898839 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 697/1412] 10.222.0.50 () {54 vars in 918 bytes} [Fri May 27 03:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:17.903881 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30c140-7014-4a27-b64f-0b4fb975efc5 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:17.904177 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30c140-7014-4a27-b64f-0b4fb975efc5 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:17.904445 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b30c140-7014-4a27-b64f-0b4fb975efc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:17.923421 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-6b30c140-7014-4a27-b64f-0b4fb975efc5 None None] MFA Rules not processed for user `1a699988f08a4001b3b7d764f01f4fc5`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:17.924631 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-6b30c140-7014-4a27-b64f-0b4fb975efc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:17.934588 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 716/1413] 10.222.0.50 () {58 vars in 1015 bytes} [Fri May 27 03:16:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:20.834004 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f287618-3abd-49dc-b03b-7f4d20774776 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:20.834888 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f287618-3abd-49dc-b03b-7f4d20774776 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:20.834888 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f287618-3abd-49dc-b03b-7f4d20774776 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:20.837119 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 698/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:16:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:20.844467 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6926983b-51b7-4156-b322-f98fa617f03a None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:20.845332 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6926983b-51b7-4156-b322-f98fa617f03a None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:20.845624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6926983b-51b7-4156-b322-f98fa617f03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:20.869958 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-6926983b-51b7-4156-b322-f98fa617f03a None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:20.871176 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-6926983b-51b7-4156-b322-f98fa617f03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:20.879360 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 717/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:16:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:16:20.927295 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f287618-3abd-49dc-b03b-7f4d20774776 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:20.946959 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f287618-3abd-49dc-b03b-7f4d20774776 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:20.953112 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8245f17e-cc3c-400c-88f8-e4f0e712e8ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:20.953746 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245f17e-cc3c-400c-88f8-e4f0e712e8ab service nova] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:20.954033 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245f17e-cc3c-400c-88f8-e4f0e712e8ab service nova] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:20.954303 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8245f17e-cc3c-400c-88f8-e4f0e712e8ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:20.963388 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8245f17e-cc3c-400c-88f8-e4f0e712e8ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:20.965168 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8245f17e-cc3c-400c-88f8-e4f0e712e8ab service nova] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:20.974977 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 699/1416] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:16:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:16:21.948033 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd1422-db31-4e80-8e8c-a3e1ff88e1fd None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:21.949063 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd1422-db31-4e80-8e8c-a3e1ff88e1fd None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:21.949063 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd1422-db31-4e80-8e8c-a3e1ff88e1fd None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:21.951189 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 718/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:16:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:21.959684 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b537a3c-5a47-45ff-bcb9-d78afa242b60 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:21.960127 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b537a3c-5a47-45ff-bcb9-d78afa242b60 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:21.960286 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b537a3c-5a47-45ff-bcb9-d78afa242b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:21.982964 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-9b537a3c-5a47-45ff-bcb9-d78afa242b60 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:21.984595 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-9b537a3c-5a47-45ff-bcb9-d78afa242b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:21.993615 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 700/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:16:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:22.166692 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd1422-db31-4e80-8e8c-a3e1ff88e1fd None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:22.185248 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd1422-db31-4e80-8e8c-a3e1ff88e1fd None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:22.192416 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7464d5-a917-4165-b11f-d5737e8b7044 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:22.193087 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7464d5-a917-4165-b11f-d5737e8b7044 service nova] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:22.193366 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7464d5-a917-4165-b11f-d5737e8b7044 service nova] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:22.193636 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7464d5-a917-4165-b11f-d5737e8b7044 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:22.203203 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7464d5-a917-4165-b11f-d5737e8b7044 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:22.204732 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7464d5-a917-4165-b11f-d5737e8b7044 service nova] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:22.214449 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 719/1419] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:16:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:16:23.187244 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a83de-7165-49bc-bd1a-d4997475e42b None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:23.187851 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a83de-7165-49bc-bd1a-d4997475e42b None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:23.187851 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a83de-7165-49bc-bd1a-d4997475e42b None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:23.189205 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 701/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:16:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:23.197595 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b14d6-a4ec-4f93-8c8c-f3c1213e248f None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:23.198037 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b14d6-a4ec-4f93-8c8c-f3c1213e248f None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:23.198236 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16b14d6-a4ec-4f93-8c8c-f3c1213e248f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:23.222902 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-b16b14d6-a4ec-4f93-8c8c-f3c1213e248f None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:23.224117 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-b16b14d6-a4ec-4f93-8c8c-f3c1213e248f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:23.232322 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 720/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:23.387739 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a83de-7165-49bc-bd1a-d4997475e42b None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:23.410147 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a83de-7165-49bc-bd1a-d4997475e42b None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:23.415965 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1d0b9-c37c-402f-a771-e2e6394fc7ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:23.416608 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1d0b9-c37c-402f-a771-e2e6394fc7ad service nova] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:23.416889 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1d0b9-c37c-402f-a771-e2e6394fc7ad service nova] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:23.417197 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1d0b9-c37c-402f-a771-e2e6394fc7ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:23.425922 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea1d0b9-c37c-402f-a771-e2e6394fc7ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:23.427520 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea1d0b9-c37c-402f-a771-e2e6394fc7ad service nova] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:23.437197 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 702/1422] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:16:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:16:24.406130 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af1e00-bfff-40e8-9896-e3b5050337ef None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:24.406701 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af1e00-bfff-40e8-9896-e3b5050337ef None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:24.406701 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af1e00-bfff-40e8-9896-e3b5050337ef None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:24.408443 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 721/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:16:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:24.414870 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6bb65-210a-48dc-a1f2-4f136c30d0c6 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:24.415620 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6bb65-210a-48dc-a1f2-4f136c30d0c6 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:24.415794 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc6bb65-210a-48dc-a1f2-4f136c30d0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:24.438082 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-bdc6bb65-210a-48dc-a1f2-4f136c30d0c6 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:24.439284 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-bdc6bb65-210a-48dc-a1f2-4f136c30d0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:24.450605 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 703/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:16:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:24.599095 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af1e00-bfff-40e8-9896-e3b5050337ef None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:24.619746 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af1e00-bfff-40e8-9896-e3b5050337ef None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:24.626275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b33cde-794b-4e79-8aeb-609dbd320a31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:24.626932 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b33cde-794b-4e79-8aeb-609dbd320a31 service nova] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:24.627216 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b33cde-794b-4e79-8aeb-609dbd320a31 service nova] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:24.627492 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b33cde-794b-4e79-8aeb-609dbd320a31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:24.636147 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b33cde-794b-4e79-8aeb-609dbd320a31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:24.637785 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b33cde-794b-4e79-8aeb-609dbd320a31 service nova] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:24.647920 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 722/1425] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:16:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:16:25.564196 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299eceb-4e7b-4488-9850-8d47229ad73f None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:25.565074 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299eceb-4e7b-4488-9850-8d47229ad73f None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:25.565074 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4299eceb-4e7b-4488-9850-8d47229ad73f None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:25.567178 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 704/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:16:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:25.573334 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93290ccf-5bd8-4bc7-924c-c9e398784b86 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:25.574020 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93290ccf-5bd8-4bc7-924c-c9e398784b86 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:25.574198 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93290ccf-5bd8-4bc7-924c-c9e398784b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:25.595581 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-93290ccf-5bd8-4bc7-924c-c9e398784b86 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:25.596775 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-93290ccf-5bd8-4bc7-924c-c9e398784b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:25.605568 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 723/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:16:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:25.757788 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299eceb-4e7b-4488-9850-8d47229ad73f None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:25.776439 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4299eceb-4e7b-4488-9850-8d47229ad73f None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:25.783284 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d420d05-a582-4970-8a16-b255f8752b3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:25.783935 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d420d05-a582-4970-8a16-b255f8752b3b service nova] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:25.784218 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d420d05-a582-4970-8a16-b255f8752b3b service nova] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:25.784497 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d420d05-a582-4970-8a16-b255f8752b3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:25.793602 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d420d05-a582-4970-8a16-b255f8752b3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:25.795207 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d420d05-a582-4970-8a16-b255f8752b3b service nova] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:25.804851 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 705/1428] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:16:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:16:26.654848 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf006d-80e5-4c55-8b9b-cdc29291247b None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:26.655414 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf006d-80e5-4c55-8b9b-cdc29291247b None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:26.655414 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bf006d-80e5-4c55-8b9b-cdc29291247b None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:26.656983 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 724/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:16:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:16:26.663278 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362292eb-97d5-4980-956d-c559d6cf9fb3 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:26.663748 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362292eb-97d5-4980-956d-c559d6cf9fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:26.663844 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362292eb-97d5-4980-956d-c559d6cf9fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:26.692426 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-362292eb-97d5-4980-956d-c559d6cf9fb3 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:26.694374 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-362292eb-97d5-4980-956d-c559d6cf9fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:26.706876 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 706/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:16:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:26.862339 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bf006d-80e5-4c55-8b9b-cdc29291247b None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:26.887096 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bf006d-80e5-4c55-8b9b-cdc29291247b None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:26.896267 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d9bf1a-96b7-404b-b0d0-060fb6b6fe5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:26.896905 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9bf1a-96b7-404b-b0d0-060fb6b6fe5d service nova] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:26.897222 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9bf1a-96b7-404b-b0d0-060fb6b6fe5d service nova] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:26.897493 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9bf1a-96b7-404b-b0d0-060fb6b6fe5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:26.907716 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d9bf1a-96b7-404b-b0d0-060fb6b6fe5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:26.909336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d9bf1a-96b7-404b-b0d0-060fb6b6fe5d service nova] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:26.919278 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 725/1431] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:16:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:16:27.768800 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d5194-b768-49be-84f5-ce17218f9a5d None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:27.769995 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d5194-b768-49be-84f5-ce17218f9a5d None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:27.769995 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d5194-b768-49be-84f5-ce17218f9a5d None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:27.772227 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 707/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:16:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:27.781409 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd857d3-c538-419e-a8fc-f940be566315 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:27.782102 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd857d3-c538-419e-a8fc-f940be566315 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:27.782388 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd857d3-c538-419e-a8fc-f940be566315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:27.803225 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-6fd857d3-c538-419e-a8fc-f940be566315 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:27.804431 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-6fd857d3-c538-419e-a8fc-f940be566315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:27.814534 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 726/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:16:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:27.970490 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549d5194-b768-49be-84f5-ce17218f9a5d None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:27.986673 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549d5194-b768-49be-84f5-ce17218f9a5d None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:27.992396 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ed858-75aa-4240-8eab-18c641449f0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:27.993060 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ed858-75aa-4240-8eab-18c641449f0b service nova] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:27.993341 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ed858-75aa-4240-8eab-18c641449f0b service nova] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:27.993612 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ed858-75aa-4240-8eab-18c641449f0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:28.002016 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7ed858-75aa-4240-8eab-18c641449f0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:28.003615 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7ed858-75aa-4240-8eab-18c641449f0b service nova] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:28.012982 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 708/1434] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:16:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:16:28.862216 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c38a6-ef28-45f3-99e0-a4daf942223b None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:28.862783 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c38a6-ef28-45f3-99e0-a4daf942223b None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:28.862783 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957c38a6-ef28-45f3-99e0-a4daf942223b None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:28.864132 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 727/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:16:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 03:16:28.870393 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62180c09-fcf0-40fd-83ae-b1e25a60791e None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:28.871203 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62180c09-fcf0-40fd-83ae-b1e25a60791e None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:28.871393 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62180c09-fcf0-40fd-83ae-b1e25a60791e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:28.898567 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-62180c09-fcf0-40fd-83ae-b1e25a60791e None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:28.899779 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-62180c09-fcf0-40fd-83ae-b1e25a60791e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:28.916342 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 709/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:16:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:29.069730 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c38a6-ef28-45f3-99e0-a4daf942223b None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:29.085989 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957c38a6-ef28-45f3-99e0-a4daf942223b None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:29.092643 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a039c82-5458-44bc-aa82-38e578a16409 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:29.093336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a039c82-5458-44bc-aa82-38e578a16409 service nova] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:29.093834 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a039c82-5458-44bc-aa82-38e578a16409 service nova] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:29.094485 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a039c82-5458-44bc-aa82-38e578a16409 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:29.103734 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a039c82-5458-44bc-aa82-38e578a16409 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:29.105423 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a039c82-5458-44bc-aa82-38e578a16409 service nova] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:29.115597 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 728/1437] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:16:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:16:29.965976 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e56e02-e96a-4ddc-bbed-586587960da5 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:29.966595 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e56e02-e96a-4ddc-bbed-586587960da5 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:29.966595 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e56e02-e96a-4ddc-bbed-586587960da5 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:29.967915 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 710/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:29.974407 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daad35b-3dde-4db2-a9a6-2bdedf155324 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:29.975231 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daad35b-3dde-4db2-a9a6-2bdedf155324 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:29.975231 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daad35b-3dde-4db2-a9a6-2bdedf155324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:30.003564 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-5daad35b-3dde-4db2-a9a6-2bdedf155324 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:30.004755 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-5daad35b-3dde-4db2-a9a6-2bdedf155324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:30.014238 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 729/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:16:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:30.169984 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e56e02-e96a-4ddc-bbed-586587960da5 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:30.188728 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e56e02-e96a-4ddc-bbed-586587960da5 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:30.194623 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada24f6-1931-4d09-a4a2-a4433ef67f4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:30.195257 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada24f6-1931-4d09-a4a2-a4433ef67f4c service nova] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:30.195535 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada24f6-1931-4d09-a4a2-a4433ef67f4c service nova] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:30.195802 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada24f6-1931-4d09-a4a2-a4433ef67f4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:30.204328 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bada24f6-1931-4d09-a4a2-a4433ef67f4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:30.205925 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bada24f6-1931-4d09-a4a2-a4433ef67f4c service nova] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:30.215362 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 711/1440] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:16:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:16:32.279495 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea9cdf-8cc0-43d3-97d8-0c2723529494 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:32.280052 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea9cdf-8cc0-43d3-97d8-0c2723529494 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:32.280052 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea9cdf-8cc0-43d3-97d8-0c2723529494 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:32.281569 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 730/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:16:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:32.287038 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54e8b9-d86d-4709-bb3e-13244233b180 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:32.287486 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54e8b9-d86d-4709-bb3e-13244233b180 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:32.287593 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54e8b9-d86d-4709-bb3e-13244233b180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:32.306393 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-4d54e8b9-d86d-4709-bb3e-13244233b180 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:32.307598 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-4d54e8b9-d86d-4709-bb3e-13244233b180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:32.316309 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 712/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:16:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:32.357615 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea9cdf-8cc0-43d3-97d8-0c2723529494 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:32.373737 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea9cdf-8cc0-43d3-97d8-0c2723529494 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:32.378555 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee42e81-45fe-4fd4-916b-e85251875c39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:32.379191 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee42e81-45fe-4fd4-916b-e85251875c39 service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:32.379466 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee42e81-45fe-4fd4-916b-e85251875c39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:32.379732 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee42e81-45fe-4fd4-916b-e85251875c39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:32.389308 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee42e81-45fe-4fd4-916b-e85251875c39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:32.390823 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee42e81-45fe-4fd4-916b-e85251875c39 service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:32.400816 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 731/1443] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:16:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:16:34.768073 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d276c31-2d07-4a4c-b062-acebf5ffe1bc None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:34.768704 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d276c31-2d07-4a4c-b062-acebf5ffe1bc None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:34.768704 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d276c31-2d07-4a4c-b062-acebf5ffe1bc None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:34.770025 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 713/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:34.776727 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac1dd9e-0df8-4104-86e5-2245fcae5952 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:34.777536 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac1dd9e-0df8-4104-86e5-2245fcae5952 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:34.777718 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac1dd9e-0df8-4104-86e5-2245fcae5952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:34.799070 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-2ac1dd9e-0df8-4104-86e5-2245fcae5952 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:34.800242 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-2ac1dd9e-0df8-4104-86e5-2245fcae5952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:34.808718 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 732/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:16:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:34.849855 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d276c31-2d07-4a4c-b062-acebf5ffe1bc None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:34.862669 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d276c31-2d07-4a4c-b062-acebf5ffe1bc None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:34.866934 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40498be-1ff1-44a5-91f1-7fc7911cb568 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:34.867576 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40498be-1ff1-44a5-91f1-7fc7911cb568 service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:34.867842 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40498be-1ff1-44a5-91f1-7fc7911cb568 service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:34.868110 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40498be-1ff1-44a5-91f1-7fc7911cb568 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:34.878064 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40498be-1ff1-44a5-91f1-7fc7911cb568 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:34.879510 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40498be-1ff1-44a5-91f1-7fc7911cb568 service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:34.889506 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 714/1446] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:16:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:16:37.527736 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7abc5c-eb0f-4ffb-aac4-d068f8929003 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:37.528702 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7abc5c-eb0f-4ffb-aac4-d068f8929003 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:37.528702 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7abc5c-eb0f-4ffb-aac4-d068f8929003 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:37.530798 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 733/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:16:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:37.536648 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95d5e9-e32f-4cb5-9a63-5c8efb34e21e None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:37.537157 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95d5e9-e32f-4cb5-9a63-5c8efb34e21e None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:37.537264 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c95d5e9-e32f-4cb5-9a63-5c8efb34e21e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:37.557425 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-8c95d5e9-e32f-4cb5-9a63-5c8efb34e21e None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:37.558550 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-8c95d5e9-e32f-4cb5-9a63-5c8efb34e21e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:37.567463 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 715/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:16:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:37.605609 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7abc5c-eb0f-4ffb-aac4-d068f8929003 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:37.617647 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7abc5c-eb0f-4ffb-aac4-d068f8929003 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:37.622187 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed82f086-d3d8-471b-a57e-6b638bfd9f85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:37.622808 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82f086-d3d8-471b-a57e-6b638bfd9f85 service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:37.623090 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82f086-d3d8-471b-a57e-6b638bfd9f85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:37.623437 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82f086-d3d8-471b-a57e-6b638bfd9f85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:37.633443 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed82f086-d3d8-471b-a57e-6b638bfd9f85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:37.634890 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed82f086-d3d8-471b-a57e-6b638bfd9f85 service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:37.644606 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 734/1449] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:16:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:16:39.972468 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cf6e5-aa90-4f0c-9e1e-195bb5e4aec6 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:39.973095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cf6e5-aa90-4f0c-9e1e-195bb5e4aec6 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:39.973095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085cf6e5-aa90-4f0c-9e1e-195bb5e4aec6 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:39.974421 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 716/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:16:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:39.980916 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90072b7-b33c-48b2-89f1-f97d80a3d119 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:39.981665 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90072b7-b33c-48b2-89f1-f97d80a3d119 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:39.981824 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90072b7-b33c-48b2-89f1-f97d80a3d119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:40.001933 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-d90072b7-b33c-48b2-89f1-f97d80a3d119 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:40.003144 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-d90072b7-b33c-48b2-89f1-f97d80a3d119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:40.012725 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 735/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:16:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:40.052394 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085cf6e5-aa90-4f0c-9e1e-195bb5e4aec6 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:40.063767 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085cf6e5-aa90-4f0c-9e1e-195bb5e4aec6 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:40.067510 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ce07e0-a25e-4402-8351-df07c2b6efdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:40.068151 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce07e0-a25e-4402-8351-df07c2b6efdd service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:40.068427 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce07e0-a25e-4402-8351-df07c2b6efdd service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:40.068700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce07e0-a25e-4402-8351-df07c2b6efdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:40.078419 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ce07e0-a25e-4402-8351-df07c2b6efdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:40.079860 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ce07e0-a25e-4402-8351-df07c2b6efdd service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:40.090028 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 717/1452] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:16:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:16:42.387419 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7aaf2a-36b4-4d78-b3e4-2c2dd8aab1d5 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:42.388012 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7aaf2a-36b4-4d78-b3e4-2c2dd8aab1d5 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:42.388012 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7aaf2a-36b4-4d78-b3e4-2c2dd8aab1d5 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:42.389418 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 736/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:42.396426 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92651c9f-eead-4d67-a770-1f52ea91b40a None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:42.397151 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92651c9f-eead-4d67-a770-1f52ea91b40a None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:42.397329 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92651c9f-eead-4d67-a770-1f52ea91b40a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:42.424693 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-92651c9f-eead-4d67-a770-1f52ea91b40a None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:42.425959 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-92651c9f-eead-4d67-a770-1f52ea91b40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:42.435013 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 718/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:16:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:42.475438 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7aaf2a-36b4-4d78-b3e4-2c2dd8aab1d5 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:42.488747 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7aaf2a-36b4-4d78-b3e4-2c2dd8aab1d5 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:42.492377 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ec3790-163e-45b1-8b58-eb9cda96626b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:42.493037 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec3790-163e-45b1-8b58-eb9cda96626b service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:42.493313 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec3790-163e-45b1-8b58-eb9cda96626b service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:42.493583 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec3790-163e-45b1-8b58-eb9cda96626b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:42.503087 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ec3790-163e-45b1-8b58-eb9cda96626b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:42.504527 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ec3790-163e-45b1-8b58-eb9cda96626b service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:42.513944 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 737/1455] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:16:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:16:44.850425 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1f164-627f-4886-b370-2b6ec21eade5 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:44.851339 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1f164-627f-4886-b370-2b6ec21eade5 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:44.851339 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1f164-627f-4886-b370-2b6ec21eade5 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:44.853547 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 719/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:16:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:44.859842 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede2b08-4580-43ed-9848-a53bdde60ed2 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:44.860282 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede2b08-4580-43ed-9848-a53bdde60ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:44.860395 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede2b08-4580-43ed-9848-a53bdde60ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:44.879516 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-3ede2b08-4580-43ed-9848-a53bdde60ed2 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:44.880708 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-3ede2b08-4580-43ed-9848-a53bdde60ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:44.889456 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 738/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:16:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:44.930878 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba1f164-627f-4886-b370-2b6ec21eade5 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:44.943507 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba1f164-627f-4886-b370-2b6ec21eade5 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:44.948014 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331be601-3c9f-4b02-bbed-d784e4f7a86a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:44.948797 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331be601-3c9f-4b02-bbed-d784e4f7a86a service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:44.949121 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331be601-3c9f-4b02-bbed-d784e4f7a86a service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:44.949393 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331be601-3c9f-4b02-bbed-d784e4f7a86a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:44.959903 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331be601-3c9f-4b02-bbed-d784e4f7a86a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:44.961389 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331be601-3c9f-4b02-bbed-d784e4f7a86a service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:44.971641 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 720/1458] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:16:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:16:47.219202 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9f916-5b00-43e1-85f0-01724c8bab27 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:47.219810 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9f916-5b00-43e1-85f0-01724c8bab27 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:47.219810 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9f916-5b00-43e1-85f0-01724c8bab27 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:47.221168 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 739/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:16:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:47.227791 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842727bf-d817-49d8-bae0-406e74e6f747 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:47.228490 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842727bf-d817-49d8-bae0-406e74e6f747 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:47.228642 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842727bf-d817-49d8-bae0-406e74e6f747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:47.239402 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=70 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:47.256199 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-842727bf-d817-49d8-bae0-406e74e6f747 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:47.257440 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-842727bf-d817-49d8-bae0-406e74e6f747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:47.266625 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 721/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:16:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:47.305155 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a9f916-5b00-43e1-85f0-01724c8bab27 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:47.313946 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=73 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:16:47.320388 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a9f916-5b00-43e1-85f0-01724c8bab27 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:47.324301 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d7040-d8be-487b-8e53-1b721f060197 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:47.324969 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d7040-d8be-487b-8e53-1b721f060197 service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:47.325244 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d7040-d8be-487b-8e53-1b721f060197 service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:47.325515 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d7040-d8be-487b-8e53-1b721f060197 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:47.335119 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8d7040-d8be-487b-8e53-1b721f060197 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:47.336564 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8d7040-d8be-487b-8e53-1b721f060197 service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:47.346058 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 740/1461] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:16:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:16:49.618120 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499db37-fde5-4074-b34b-02c5a95c9b1f None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:49.618120 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499db37-fde5-4074-b34b-02c5a95c9b1f None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:49.618120 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499db37-fde5-4074-b34b-02c5a95c9b1f None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:49.619984 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 722/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:16:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:49.626244 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff38a3-cef1-48ce-bd10-a4e43820372f None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:49.626784 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff38a3-cef1-48ce-bd10-a4e43820372f None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:49.626784 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff38a3-cef1-48ce-bd10-a4e43820372f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:49.656933 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-ceff38a3-cef1-48ce-bd10-a4e43820372f None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:49.658174 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-ceff38a3-cef1-48ce-bd10-a4e43820372f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:49.667598 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 741/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:16:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:49.709789 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b499db37-fde5-4074-b34b-02c5a95c9b1f None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:49.725226 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b499db37-fde5-4074-b34b-02c5a95c9b1f None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:49.729366 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6dd96-3711-49d7-8bb8-070bdb9b2617 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:49.730004 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6dd96-3711-49d7-8bb8-070bdb9b2617 service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:49.730279 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6dd96-3711-49d7-8bb8-070bdb9b2617 service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:49.730551 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6dd96-3711-49d7-8bb8-070bdb9b2617 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:49.739940 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb6dd96-3711-49d7-8bb8-070bdb9b2617 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:49.741411 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb6dd96-3711-49d7-8bb8-070bdb9b2617 service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:49.750711 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 723/1464] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:16:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:16:51.979023 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba343d8-3958-4f14-a46c-bc12e61b858f None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:51.979666 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba343d8-3958-4f14-a46c-bc12e61b858f None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:51.979666 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba343d8-3958-4f14-a46c-bc12e61b858f None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:51.980968 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 742/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:16:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:51.988234 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e43a29-44ca-4294-8736-5d16e2d07b9e None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:51.988716 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e43a29-44ca-4294-8736-5d16e2d07b9e None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:51.988817 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e43a29-44ca-4294-8736-5d16e2d07b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:52.011856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-65e43a29-44ca-4294-8736-5d16e2d07b9e None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:52.013091 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-65e43a29-44ca-4294-8736-5d16e2d07b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:52.023250 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 724/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:16:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:52.063259 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba343d8-3958-4f14-a46c-bc12e61b858f None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:52.079967 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba343d8-3958-4f14-a46c-bc12e61b858f None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:52.083984 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b30c1-c59e-4e04-93aa-394ddc0b3e91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:52.084628 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b30c1-c59e-4e04-93aa-394ddc0b3e91 service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:52.084907 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b30c1-c59e-4e04-93aa-394ddc0b3e91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:52.085207 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b30c1-c59e-4e04-93aa-394ddc0b3e91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:52.095077 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3b30c1-c59e-4e04-93aa-394ddc0b3e91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:52.096527 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3b30c1-c59e-4e04-93aa-394ddc0b3e91 service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:52.106358 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 743/1467] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:16:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:16:54.319454 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fde1e-388e-4136-926e-0af54e69ceb5 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:54.319924 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fde1e-388e-4136-926e-0af54e69ceb5 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:54.320016 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fde1e-388e-4136-926e-0af54e69ceb5 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:54.321440 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 725/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:16:54.327784 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fbe9b8-e580-41e2-987b-ef2740a75ae9 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:54.328422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fbe9b8-e580-41e2-987b-ef2740a75ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:54.328727 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fbe9b8-e580-41e2-987b-ef2740a75ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:54.357773 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-c6fbe9b8-e580-41e2-987b-ef2740a75ae9 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:16:54.359366 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-c6fbe9b8-e580-41e2-987b-ef2740a75ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:16:54.370655 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 744/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:16:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:16:54.407868 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791fde1e-388e-4136-926e-0af54e69ceb5 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:16:54.420354 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791fde1e-388e-4136-926e-0af54e69ceb5 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:16:54.424434 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a126496f-4d95-45fe-b5be-5130cc044ae4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:16:54.425087 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126496f-4d95-45fe-b5be-5130cc044ae4 service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:16:54.425395 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126496f-4d95-45fe-b5be-5130cc044ae4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:16:54.425660 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126496f-4d95-45fe-b5be-5130cc044ae4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:16:54.435386 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a126496f-4d95-45fe-b5be-5130cc044ae4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:16:54.436843 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a126496f-4d95-45fe-b5be-5130cc044ae4 service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:16:54.447102 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 726/1470] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:16:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:17:04.349302 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=17 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:17:04.442351 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=17 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:17:27.777356 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023138b-1027-4d22-8fde-c7b1a8e487da None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:27.777951 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023138b-1027-4d22-8fde-c7b1a8e487da None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:27.777951 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7023138b-1027-4d22-8fde-c7b1a8e487da None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:27.779401 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 745/1471] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:17:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:17:27.785874 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10092034-abe3-4f8e-8fed-9598e7d215d3 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:27.786640 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10092034-abe3-4f8e-8fed-9598e7d215d3 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:27.786789 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10092034-abe3-4f8e-8fed-9598e7d215d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:27.815748 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-10092034-abe3-4f8e-8fed-9598e7d215d3 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:17:27.816979 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-10092034-abe3-4f8e-8fed-9598e7d215d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:17:27.827168 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 727/1472] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:17:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:17:27.883304 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7fc4db-a5a4-4af7-ad16-d0cb1f14573c None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:27.883643 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7fc4db-a5a4-4af7-ad16-d0cb1f14573c None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:27.883976 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7fc4db-a5a4-4af7-ad16-d0cb1f14573c None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:27.886181 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 746/1473] 10.222.0.50 () {54 vars in 962 bytes} [Fri May 27 03:17:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:17:27.891745 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f23023-93ff-4d11-a282-d9d1108b718a None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:27.892009 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f23023-93ff-4d11-a282-d9d1108b718a None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:27.892310 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f23023-93ff-4d11-a282-d9d1108b718a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:27.912761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-e9f23023-93ff-4d11-a282-d9d1108b718a None None] MFA Rules not processed for user `5b3c9c5c8d2e4376b1b70d974d4cb851`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:17:27.914038 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-e9f23023-93ff-4d11-a282-d9d1108b718a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:17:27.921467 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 728/1474] 10.222.0.50 () {58 vars in 1059 bytes} [Fri May 27 03:17:27 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:17:27.928939 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7fc4db-a5a4-4af7-ad16-d0cb1f14573c None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:27.940891 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7fc4db-a5a4-4af7-ad16-d0cb1f14573c None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:27.945833 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1440324-19e9-4680-85d9-5ea1f6acd1ba service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3c9c5c8d2e4376b1b70d974d4cb851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:27.946481 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1440324-19e9-4680-85d9-5ea1f6acd1ba service glance] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:27.946759 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1440324-19e9-4680-85d9-5ea1f6acd1ba service glance] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:27.947033 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1440324-19e9-4680-85d9-5ea1f6acd1ba service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:27.959314 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1440324-19e9-4680-85d9-5ea1f6acd1ba service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:27.961493 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1440324-19e9-4680-85d9-5ea1f6acd1ba service glance] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:27.973979 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 747/1475] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:17:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:17:30.266720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471b7a2-18f3-4b96-a057-2c41d2448af1 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:30.267689 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471b7a2-18f3-4b96-a057-2c41d2448af1 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:30.267689 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471b7a2-18f3-4b96-a057-2c41d2448af1 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:30.269794 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 729/1476] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:17:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:17:30.279084 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62b603-167b-460f-b601-ced338c90446 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:30.279571 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62b603-167b-460f-b601-ced338c90446 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:30.279757 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62b603-167b-460f-b601-ced338c90446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:30.311566 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-5e62b603-167b-460f-b601-ced338c90446 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:17:30.313508 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-5e62b603-167b-460f-b601-ced338c90446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:17:30.327865 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 748/1477] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:17:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:17:30.351121 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471b7a2-18f3-4b96-a057-2c41d2448af1 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:30.365146 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471b7a2-18f3-4b96-a057-2c41d2448af1 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:30.370022 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12beba5-5170-4e19-96ca-74e3e5660952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:30.370671 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12beba5-5170-4e19-96ca-74e3e5660952 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:30.370948 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12beba5-5170-4e19-96ca-74e3e5660952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:30.371220 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12beba5-5170-4e19-96ca-74e3e5660952 admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:30.371939 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12beba5-5170-4e19-96ca-74e3e5660952 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:30.374169 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: WARNING py.warnings [None req-e12beba5-5170-4e19-96ca-74e3e5660952 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:17:30.374169 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: warnings.warn(msg) May 27 03:17:30.374169 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: May 27 03:17:30.375313 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12beba5-5170-4e19-96ca-74e3e5660952 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:30.395057 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 730/1478] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 03:17:30 2022] GET /identity/v3/projects => generated 2553 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:17:32.647729 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2392dc6-7d9e-4a34-8e1b-b33b4c0261f3 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:32.648340 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2392dc6-7d9e-4a34-8e1b-b33b4c0261f3 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:32.648340 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2392dc6-7d9e-4a34-8e1b-b33b4c0261f3 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:32.650058 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 749/1479] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:17:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:17:32.656655 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a579f-56b1-44d1-addb-bdc9d513b065 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:32.657396 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a579f-56b1-44d1-addb-bdc9d513b065 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:32.657528 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a579f-56b1-44d1-addb-bdc9d513b065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:32.681448 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-057a579f-56b1-44d1-addb-bdc9d513b065 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:17:32.682658 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-057a579f-56b1-44d1-addb-bdc9d513b065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:17:32.691950 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 731/1480] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:17:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:17:32.739265 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2392dc6-7d9e-4a34-8e1b-b33b4c0261f3 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:32.764695 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2392dc6-7d9e-4a34-8e1b-b33b4c0261f3 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:32.771883 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cc8cbb-814b-4323-ab85-7c7423f79818 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:32.772546 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cc8cbb-814b-4323-ab85-7c7423f79818 service nova] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:32.772855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cc8cbb-814b-4323-ab85-7c7423f79818 service nova] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:32.773146 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cc8cbb-814b-4323-ab85-7c7423f79818 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:32.783595 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cc8cbb-814b-4323-ab85-7c7423f79818 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:32.785213 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cc8cbb-814b-4323-ab85-7c7423f79818 service nova] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:32.795560 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 750/1481] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:17:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:17:35.175945 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb28677-e206-43bc-97be-3b71765a1575 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:35.176551 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb28677-e206-43bc-97be-3b71765a1575 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:35.176551 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb28677-e206-43bc-97be-3b71765a1575 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:35.177932 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 732/1482] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:17:35.183328 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c1be9-a9c6-4450-9067-76b36a4cf698 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:35.183752 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c1be9-a9c6-4450-9067-76b36a4cf698 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:35.183854 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c1be9-a9c6-4450-9067-76b36a4cf698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:35.203738 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-630c1be9-a9c6-4450-9067-76b36a4cf698 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:17:35.204838 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-630c1be9-a9c6-4450-9067-76b36a4cf698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:17:35.214232 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 751/1483] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:17:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:17:35.253791 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb28677-e206-43bc-97be-3b71765a1575 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:35.269533 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb28677-e206-43bc-97be-3b71765a1575 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:35.273703 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfebeb7-8bec-4fd0-abf1-ea672d42ad1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:35.274694 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfebeb7-8bec-4fd0-abf1-ea672d42ad1c service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:35.274968 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfebeb7-8bec-4fd0-abf1-ea672d42ad1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:35.275235 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfebeb7-8bec-4fd0-abf1-ea672d42ad1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:35.285416 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfebeb7-8bec-4fd0-abf1-ea672d42ad1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:35.286854 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfebeb7-8bec-4fd0-abf1-ea672d42ad1c service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:35.296840 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 733/1484] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:17:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:17:37.603374 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3b7bc-c34e-4378-9819-c24bd4efad86 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:37.603928 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3b7bc-c34e-4378-9819-c24bd4efad86 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:37.603928 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3b7bc-c34e-4378-9819-c24bd4efad86 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:37.605264 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 752/1485] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:17:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:17:37.612033 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69045c9d-adeb-444c-beee-0a18082b8240 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:37.612813 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69045c9d-adeb-444c-beee-0a18082b8240 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:37.612996 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69045c9d-adeb-444c-beee-0a18082b8240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:37.640913 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-69045c9d-adeb-444c-beee-0a18082b8240 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:17:37.642132 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-69045c9d-adeb-444c-beee-0a18082b8240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:17:37.651318 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 734/1486] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:17:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:17:37.689817 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3b7bc-c34e-4378-9819-c24bd4efad86 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:37.705324 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3b7bc-c34e-4378-9819-c24bd4efad86 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:37.709261 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b647eb-75fe-4576-81ba-51811d1e9cfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:37.709905 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b647eb-75fe-4576-81ba-51811d1e9cfc service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:37.710182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b647eb-75fe-4576-81ba-51811d1e9cfc service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:37.710457 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b647eb-75fe-4576-81ba-51811d1e9cfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:37.720251 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b647eb-75fe-4576-81ba-51811d1e9cfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:37.721741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b647eb-75fe-4576-81ba-51811d1e9cfc service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:37.731091 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 753/1487] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:17:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:17:40.180048 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ee23d-0908-4717-844c-20207d2fddb1 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:40.180671 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ee23d-0908-4717-844c-20207d2fddb1 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:40.180671 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ee23d-0908-4717-844c-20207d2fddb1 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:40.182019 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 735/1488] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:17:40.188557 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21737dfc-8f2a-4a68-92e1-0075f5b5468f None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:40.189029 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21737dfc-8f2a-4a68-92e1-0075f5b5468f None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:40.189128 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21737dfc-8f2a-4a68-92e1-0075f5b5468f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:40.209124 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-21737dfc-8f2a-4a68-92e1-0075f5b5468f None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:17:40.210326 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-21737dfc-8f2a-4a68-92e1-0075f5b5468f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:17:40.219093 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 754/1489] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:17:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:17:40.262733 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ee23d-0908-4717-844c-20207d2fddb1 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:40.281592 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ee23d-0908-4717-844c-20207d2fddb1 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:40.285978 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e05d8fc-6a51-4db9-806b-4b43fec71983 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:40.286631 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e05d8fc-6a51-4db9-806b-4b43fec71983 service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:40.286910 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e05d8fc-6a51-4db9-806b-4b43fec71983 service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:40.287184 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e05d8fc-6a51-4db9-806b-4b43fec71983 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:40.297666 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e05d8fc-6a51-4db9-806b-4b43fec71983 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:40.299041 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e05d8fc-6a51-4db9-806b-4b43fec71983 service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:40.309221 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 736/1490] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:17:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:17:42.704416 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88091f20-1364-48f7-8d5e-6ad39ac2a9d9 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:42.705011 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88091f20-1364-48f7-8d5e-6ad39ac2a9d9 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:42.705011 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88091f20-1364-48f7-8d5e-6ad39ac2a9d9 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:42.706419 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 755/1491] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 03:17:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:17:42.712809 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81707e11-6905-46e2-ba48-fa282a65ae14 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:42.713312 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81707e11-6905-46e2-ba48-fa282a65ae14 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:42.713411 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81707e11-6905-46e2-ba48-fa282a65ae14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:42.738357 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-81707e11-6905-46e2-ba48-fa282a65ae14 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:17:42.739625 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-81707e11-6905-46e2-ba48-fa282a65ae14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:17:42.748104 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 737/1492] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 03:17:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:17:42.790933 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88091f20-1364-48f7-8d5e-6ad39ac2a9d9 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:17:42.804764 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88091f20-1364-48f7-8d5e-6ad39ac2a9d9 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:17:42.808977 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d740cdf-bcdb-4f5f-a88a-4b089d91597d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:17:42.809880 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d740cdf-bcdb-4f5f-a88a-4b089d91597d service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:17:42.810161 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d740cdf-bcdb-4f5f-a88a-4b089d91597d service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:17:42.810433 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d740cdf-bcdb-4f5f-a88a-4b089d91597d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:17:42.819584 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d740cdf-bcdb-4f5f-a88a-4b089d91597d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:17:42.821051 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d740cdf-bcdb-4f5f-a88a-4b089d91597d service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:17:42.830619 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 756/1493] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:17:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:17:52.732945 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=33 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:17:52.826209 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=33 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:19:24.108239 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4222d-c04f-4f7d-9f72-b0da1a63c34c None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:19:24.108851 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4222d-c04f-4f7d-9f72-b0da1a63c34c None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:19:24.108851 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4222d-c04f-4f7d-9f72-b0da1a63c34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:19:24.128383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-56d4222d-c04f-4f7d-9f72-b0da1a63c34c None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:19:24.129612 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-56d4222d-c04f-4f7d-9f72-b0da1a63c34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:19:24.138318 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 738/1494] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:19:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:19:24.148446 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd36661-f721-4f27-ae43-d0637ff466a4 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:19:24.149043 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd36661-f721-4f27-ae43-d0637ff466a4 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:19:24.149043 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd36661-f721-4f27-ae43-d0637ff466a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:19:24.170290 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-1bd36661-f721-4f27-ae43-d0637ff466a4 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:19:24.171496 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-1bd36661-f721-4f27-ae43-d0637ff466a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:19:24.182020 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 757/1495] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:19:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:19:24.189800 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d4222d-c04f-4f7d-9f72-b0da1a63c34c None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:19:24.205810 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d4222d-c04f-4f7d-9f72-b0da1a63c34c None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:19:24.214410 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef16e4b1-d1c9-43da-a460-d1fd0e7c6388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:19:24.215068 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16e4b1-d1c9-43da-a460-d1fd0e7c6388 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:19:24.215344 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16e4b1-d1c9-43da-a460-d1fd0e7c6388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:19:24.215612 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef16e4b1-d1c9-43da-a460-d1fd0e7c6388 admin admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:19:24.216338 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef16e4b1-d1c9-43da-a460-d1fd0e7c6388 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:19:24.216587 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef16e4b1-d1c9-43da-a460-d1fd0e7c6388 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:19:24.217671 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: WARNING py.warnings [None req-ef16e4b1-d1c9-43da-a460-d1fd0e7c6388 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 27 03:19:24.217671 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: warnings.warn(msg) May 27 03:19:24.217671 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: May 27 03:19:24.218562 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef16e4b1-d1c9-43da-a460-d1fd0e7c6388 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:19:24.231340 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 739/1496] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:19:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:19:24.239668 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd36661-f721-4f27-ae43-d0637ff466a4 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:19:24.249156 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd36661-f721-4f27-ae43-d0637ff466a4 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:19:24.254555 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1773609-76ec-4cbc-9595-cc9197d1ebe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:19:24.255215 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1773609-76ec-4cbc-9595-cc9197d1ebe5 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:19:24.255489 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1773609-76ec-4cbc-9595-cc9197d1ebe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:19:24.255765 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1773609-76ec-4cbc-9595-cc9197d1ebe5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:19:24.256451 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1773609-76ec-4cbc-9595-cc9197d1ebe5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:19:24.258295 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: WARNING py.warnings [None req-d1773609-76ec-4cbc-9595-cc9197d1ebe5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:19:24.258295 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: warnings.warn(msg) May 27 03:19:24.258295 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: May 27 03:19:24.259500 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1773609-76ec-4cbc-9595-cc9197d1ebe5 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:19:24.303726 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 758/1497] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:19:24 2022] POST /identity/v3/projects => generated 377 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:19:24.308535 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef16e4b1-d1c9-43da-a460-d1fd0e7c6388 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:19:24.318725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef16e4b1-d1c9-43da-a460-d1fd0e7c6388 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:19:24.376649 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a58aaa-7abd-4445-98dd-b1e2c450c891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:19:24.377284 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a58aaa-7abd-4445-98dd-b1e2c450c891 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:19:24.377563 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a58aaa-7abd-4445-98dd-b1e2c450c891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:19:24.377836 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a58aaa-7abd-4445-98dd-b1e2c450c891 admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:19:24.378520 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a58aaa-7abd-4445-98dd-b1e2c450c891 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:19:24.379536 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: WARNING py.warnings [None req-a9a58aaa-7abd-4445-98dd-b1e2c450c891 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:19:24.379536 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: warnings.warn(msg) May 27 03:19:24.379536 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: May 27 03:19:24.380760 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a58aaa-7abd-4445-98dd-b1e2c450c891 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:19:24.405515 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 740/1498] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:19:24 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:19:24.411850 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1773609-76ec-4cbc-9595-cc9197d1ebe5 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:19:24.424377 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1773609-76ec-4cbc-9595-cc9197d1ebe5 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:19:24.429263 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b931b389-f42c-4272-926e-fd4a0d51dee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:19:24.429926 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b931b389-f42c-4272-926e-fd4a0d51dee3 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:19:24.430206 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b931b389-f42c-4272-926e-fd4a0d51dee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:19:24.430477 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b931b389-f42c-4272-926e-fd4a0d51dee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:19:24.431194 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b931b389-f42c-4272-926e-fd4a0d51dee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:19:24.432467 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: WARNING py.warnings [None req-b931b389-f42c-4272-926e-fd4a0d51dee3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 27 03:19:24.432467 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: warnings.warn(msg) May 27 03:19:24.432467 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: May 27 03:19:24.433431 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b931b389-f42c-4272-926e-fd4a0d51dee3 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:19:24.446775 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 759/1499] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:19:24.451346 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a58aaa-7abd-4445-98dd-b1e2c450c891 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:19:24.462002 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a58aaa-7abd-4445-98dd-b1e2c450c891 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:19:24.466938 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea3f376-391a-4707-a8f7-363a60f8556b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:19:24.467595 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3f376-391a-4707-a8f7-363a60f8556b admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:19:24.467875 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3f376-391a-4707-a8f7-363a60f8556b admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:19:24.468152 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea3f376-391a-4707-a8f7-363a60f8556b admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:19:24.468878 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea3f376-391a-4707-a8f7-363a60f8556b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:19:24.469909 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: WARNING py.warnings [None req-3ea3f376-391a-4707-a8f7-363a60f8556b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 27 03:19:24.469909 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: warnings.warn(msg) May 27 03:19:24.469909 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: May 27 03:19:24.470858 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea3f376-391a-4707-a8f7-363a60f8556b admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:19:24.485420 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 741/1500] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:19:24.489820 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b931b389-f42c-4272-926e-fd4a0d51dee3 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:19:24.500241 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b931b389-f42c-4272-926e-fd4a0d51dee3 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:19:24.505234 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d314fa-46c4-4be1-be23-91a19255bb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:19:24.506123 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d314fa-46c4-4be1-be23-91a19255bb81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:19:24.506396 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d314fa-46c4-4be1-be23-91a19255bb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:19:24.506662 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d314fa-46c4-4be1-be23-91a19255bb81 admin admin] PATH_INFO: `/v3/projects/df8c81a1641c48cc8e14e4abae1e5f3b/users/b7cdc8fe6d4049368e26e7bf98305fee/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:19:24.524214 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d314fa-46c4-4be1-be23-91a19255bb81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df8c81a1641c48cc8e14e4abae1e5f3b, user_id=b7cdc8fe6d4049368e26e7bf98305fee, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:19:24.525297 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: WARNING py.warnings [None req-e8d314fa-46c4-4be1-be23-91a19255bb81 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:19:24.525297 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: warnings.warn(msg) May 27 03:19:24.525297 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: May 27 03:19:24.527262 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d314fa-46c4-4be1-be23-91a19255bb81 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:19:24.543012 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 760/1501] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:19:24 2022] PUT /identity/v3/projects/df8c81a1641c48cc8e14e4abae1e5f3b/users/b7cdc8fe6d4049368e26e7bf98305fee/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:19:24.558332 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d4a91-a931-4f6d-8996-07eae1e56718 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:19:24.558626 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d4a91-a931-4f6d-8996-07eae1e56718 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:19:24.558933 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013d4a91-a931-4f6d-8996-07eae1e56718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:19:24.590224 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-013d4a91-a931-4f6d-8996-07eae1e56718 None None] MFA Rules not processed for user `b7cdc8fe6d4049368e26e7bf98305fee`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:19:24.591448 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-013d4a91-a931-4f6d-8996-07eae1e56718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:19:24.661232 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 742/1502] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:19:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:19:24.674892 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e34bc-fe8f-4030-9923-0fa3a809e432 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:19:24.675176 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e34bc-fe8f-4030-9923-0fa3a809e432 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:19:24.675452 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e34bc-fe8f-4030-9923-0fa3a809e432 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:19:24.676844 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 761/1503] 10.222.0.50 () {54 vars in 963 bytes} [Fri May 27 03:19:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:19:24.681951 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2ba1f-07a8-4c77-8933-d2aed3a96901 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:19:24.682238 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2ba1f-07a8-4c77-8933-d2aed3a96901 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:19:24.682519 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2ba1f-07a8-4c77-8933-d2aed3a96901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:19:24.701219 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-28f2ba1f-07a8-4c77-8933-d2aed3a96901 None None] MFA Rules not processed for user `36dd9b94ceb74eb4a7e9b5bbd5e8d8ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:19:24.702423 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-28f2ba1f-07a8-4c77-8933-d2aed3a96901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:19:24.748782 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:19:24.756791 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 743/1504] 10.222.0.50 () {58 vars in 1060 bytes} [Fri May 27 03:19:24 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:19:24.768590 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e34bc-fe8f-4030-9923-0fa3a809e432 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:19:24.783330 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e34bc-fe8f-4030-9923-0fa3a809e432 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:19:24.788308 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe1278-c3e4-44d2-907b-7f101d320d6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:19:24.788974 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe1278-c3e4-44d2-907b-7f101d320d6b service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:19:24.789262 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe1278-c3e4-44d2-907b-7f101d320d6b service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:19:24.789533 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe1278-c3e4-44d2-907b-7f101d320d6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:19:24.799208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fe1278-c3e4-44d2-907b-7f101d320d6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:19:24.800694 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fe1278-c3e4-44d2-907b-7f101d320d6b service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:19:24.810610 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 762/1505] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:19:24 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:19:24.854574 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c9f845-9051-43bf-ba25-2e7c91fe475d None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:19:24.854859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c9f845-9051-43bf-ba25-2e7c91fe475d None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:19:24.855132 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c9f845-9051-43bf-ba25-2e7c91fe475d None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:19:24.856462 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 744/1506] 10.222.0.50 () {52 vars in 829 bytes} [Fri May 27 03:19:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:19:24.862214 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe1278-c3e4-44d2-907b-7f101d320d6b service cinder] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:19:24.930518 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe1278-c3e4-44d2-907b-7f101d320d6b service cinder] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:19:24.936586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba5d072-aa32-44ad-8ea4-6781e06555b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:19:24.937493 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5d072-aa32-44ad-8ea4-6781e06555b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:19:24.937773 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5d072-aa32-44ad-8ea4-6781e06555b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:19:24.938042 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5d072-aa32-44ad-8ea4-6781e06555b0 admin admin] PATH_INFO: `/v3/users/b7cdc8fe6d4049368e26e7bf98305fee` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:19:24.939207 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba5d072-aa32-44ad-8ea4-6781e06555b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7cdc8fe6d4049368e26e7bf98305fee)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:19:24.940231 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: WARNING py.warnings [None req-2ba5d072-aa32-44ad-8ea4-6781e06555b0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:19:24.940231 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: warnings.warn(msg) May 27 03:19:24.940231 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: May 27 03:19:24.941473 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba5d072-aa32-44ad-8ea4-6781e06555b0 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:19:25.011370 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-2ba5d072-aa32-44ad-8ea4-6781e06555b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7cdc8fe6d4049368e26e7bf98305fee'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:19:25.012097 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-2ba5d072-aa32-44ad-8ea4-6781e06555b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7cdc8fe6d4049368e26e7bf98305fee'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:19:25.023382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-2ba5d072-aa32-44ad-8ea4-6781e06555b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7cdc8fe6d4049368e26e7bf98305fee'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:19:25.024004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-2ba5d072-aa32-44ad-8ea4-6781e06555b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7cdc8fe6d4049368e26e7bf98305fee'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:19:25.041857 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-2ba5d072-aa32-44ad-8ea4-6781e06555b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7cdc8fe6d4049368e26e7bf98305fee'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:19:25.051752 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:19:25.066266 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 763/1507] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:19:24 2022] DELETE /identity/v3/users/b7cdc8fe6d4049368e26e7bf98305fee => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:19:25.070812 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d6709e-b73d-4929-83c2-5693fbd0c409 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:19:25.071098 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d6709e-b73d-4929-83c2-5693fbd0c409 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:19:25.071375 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d6709e-b73d-4929-83c2-5693fbd0c409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:19:25.089733 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-f4d6709e-b73d-4929-83c2-5693fbd0c409 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:19:25.090942 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-f4d6709e-b73d-4929-83c2-5693fbd0c409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:19:25.151526 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 745/1508] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:19:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:19:25.161953 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba5d072-aa32-44ad-8ea4-6781e06555b0 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:19:25.163674 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-2ba5d072-aa32-44ad-8ea4-6781e06555b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:19:25.238317 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba5d072-aa32-44ad-8ea4-6781e06555b0 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:19:25.242866 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ef100-262b-4ceb-b6eb-588726c77aff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:19:25.243513 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ef100-262b-4ceb-b6eb-588726c77aff service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:19:25.243792 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ef100-262b-4ceb-b6eb-588726c77aff service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:19:25.244104 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ef100-262b-4ceb-b6eb-588726c77aff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:19:25.254432 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050ef100-262b-4ceb-b6eb-588726c77aff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:19:25.255897 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050ef100-262b-4ceb-b6eb-588726c77aff service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:19:25.266587 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 764/1509] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:19:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:19:25.657180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d6709e-b73d-4929-83c2-5693fbd0c409 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:19:25.658719 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-f4d6709e-b73d-4929-83c2-5693fbd0c409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:19:25.670457 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d6709e-b73d-4929-83c2-5693fbd0c409 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:19:25.674986 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0989070c-b230-4d2e-bf39-04603e5ffbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:19:25.675840 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989070c-b230-4d2e-bf39-04603e5ffbba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:19:25.676114 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989070c-b230-4d2e-bf39-04603e5ffbba admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:19:25.676383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989070c-b230-4d2e-bf39-04603e5ffbba admin admin] PATH_INFO: `/v3/projects/df8c81a1641c48cc8e14e4abae1e5f3b` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:19:25.677560 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0989070c-b230-4d2e-bf39-04603e5ffbba admin admin] RBAC: Authorizing `identity:delete_project(project_id=df8c81a1641c48cc8e14e4abae1e5f3b)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:19:25.678584 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: WARNING py.warnings [None req-0989070c-b230-4d2e-bf39-04603e5ffbba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:19:25.678584 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: warnings.warn(msg) May 27 03:19:25.678584 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: May 27 03:19:25.679763 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0989070c-b230-4d2e-bf39-04603e5ffbba admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:19:25.744264 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-0989070c-b230-4d2e-bf39-04603e5ffbba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df8c81a1641c48cc8e14e4abae1e5f3b', 'request_id': 'req-0989070c-b230-4d2e-bf39-04603e5ffbba'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:19:25.757770 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-0989070c-b230-4d2e-bf39-04603e5ffbba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df8c81a1641c48cc8e14e4abae1e5f3b', 'request_id': 'req-0989070c-b230-4d2e-bf39-04603e5ffbba'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:19:25.764316 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-0989070c-b230-4d2e-bf39-04603e5ffbba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df8c81a1641c48cc8e14e4abae1e5f3b', 'request_id': 'req-0989070c-b230-4d2e-bf39-04603e5ffbba'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:19:25.781651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-0989070c-b230-4d2e-bf39-04603e5ffbba admin admin] The token cache is being invalidate because project df8c81a1641c48cc8e14e4abae1e5f3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:19:25.781878 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-0989070c-b230-4d2e-bf39-04603e5ffbba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:19:25.782541 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-0989070c-b230-4d2e-bf39-04603e5ffbba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:19:25.784280 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 746/1510] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:19:25 2022] DELETE /identity/v3/projects/df8c81a1641c48cc8e14e4abae1e5f3b => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:19:35.262070 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=29 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:19:35.778442 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:21:07.995297 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fce12b-de77-4320-ba56-8fcd73f4d503 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:21:07.995928 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fce12b-de77-4320-ba56-8fcd73f4d503 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:21:07.995928 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fce12b-de77-4320-ba56-8fcd73f4d503 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:21:07.997262 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 765/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:21:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:21:08.002543 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08038b6-b349-43c6-941e-947c68f77671 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:21:08.003051 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08038b6-b349-43c6-941e-947c68f77671 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:21:08.003051 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08038b6-b349-43c6-941e-947c68f77671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:21:08.025719 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-e08038b6-b349-43c6-941e-947c68f77671 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:21:08.026866 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-e08038b6-b349-43c6-941e-947c68f77671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:21:08.095377 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 747/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:21:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:21:08.265803 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fce12b-de77-4320-ba56-8fcd73f4d503 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:21:08.267795 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-45fce12b-de77-4320-ba56-8fcd73f4d503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:21:08.356621 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fce12b-de77-4320-ba56-8fcd73f4d503 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:21:08.366843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388477f-7ac3-4122-8c23-772f56b4f6d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:21:08.367860 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388477f-7ac3-4122-8c23-772f56b4f6d7 service nova] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:21:08.368566 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388477f-7ac3-4122-8c23-772f56b4f6d7 service nova] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:21:08.369145 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388477f-7ac3-4122-8c23-772f56b4f6d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:21:08.380677 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9388477f-7ac3-4122-8c23-772f56b4f6d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:21:08.382537 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9388477f-7ac3-4122-8c23-772f56b4f6d7 service nova] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:21:08.392483 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 766/1513] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:21:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:21:08.821573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ca81f-8704-4d1d-8174-b41dfed029c9 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:21:08.821963 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ca81f-8704-4d1d-8174-b41dfed029c9 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:21:08.822316 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516ca81f-8704-4d1d-8174-b41dfed029c9 None None] PATH_INFO: `/v3` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:21:08.823711 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 748/1514] 10.222.0.67 () {54 vars in 925 bytes} [Fri May 27 03:21:08 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:21:08.831318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3527a1f-3e13-4d18-bd9b-fd873ca58377 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:21:08.831592 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3527a1f-3e13-4d18-bd9b-fd873ca58377 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:21:08.831864 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3527a1f-3e13-4d18-bd9b-fd873ca58377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:21:08.852608 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-f3527a1f-3e13-4d18-bd9b-fd873ca58377 None None] MFA Rules not processed for user `30963f2f464e4f54a8aea0175370a235`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:21:08.853869 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-f3527a1f-3e13-4d18-bd9b-fd873ca58377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:21:08.921087 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 767/1515] 10.222.0.67 () {58 vars in 1015 bytes} [Fri May 27 03:21:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:21:09.278045 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd880333-3ba4-4c23-b6c9-edcdab3be816 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:21:09.278676 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd880333-3ba4-4c23-b6c9-edcdab3be816 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:21:09.278676 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd880333-3ba4-4c23-b6c9-edcdab3be816 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:21:09.280169 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 749/1516] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:21:09.285445 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0778d-b348-49e8-a874-9642f6831615 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:21:09.285944 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0778d-b348-49e8-a874-9642f6831615 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:21:09.286040 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0778d-b348-49e8-a874-9642f6831615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:21:09.306738 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-b2e0778d-b348-49e8-a874-9642f6831615 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:21:09.307967 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-b2e0778d-b348-49e8-a874-9642f6831615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:21:09.317345 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 768/1517] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:21:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:21:09.471693 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd880333-3ba4-4c23-b6c9-edcdab3be816 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:21:09.489572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd880333-3ba4-4c23-b6c9-edcdab3be816 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:21:09.495359 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7518d133-1c2e-4602-bbe0-2f807d6e4e88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:21:09.495998 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518d133-1c2e-4602-bbe0-2f807d6e4e88 service nova] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:21:09.496293 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518d133-1c2e-4602-bbe0-2f807d6e4e88 service nova] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:21:09.496567 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7518d133-1c2e-4602-bbe0-2f807d6e4e88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:21:09.507976 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7518d133-1c2e-4602-bbe0-2f807d6e4e88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:21:09.507976 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7518d133-1c2e-4602-bbe0-2f807d6e4e88 service nova] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:21:09.517877 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 750/1518] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:21:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:21:17.995245 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b50563-3afa-4fcf-a4fa-53560b302372 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:21:17.996132 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b50563-3afa-4fcf-a4fa-53560b302372 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:21:17.996132 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b50563-3afa-4fcf-a4fa-53560b302372 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:21:17.998333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 769/1519] 10.222.0.50 () {54 vars in 960 bytes} [Fri May 27 03:21:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:21:18.003718 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e661a-eb5d-49bd-8a72-4b60471feedb None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:21:18.004153 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e661a-eb5d-49bd-8a72-4b60471feedb None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:21:18.004256 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e661a-eb5d-49bd-8a72-4b60471feedb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:21:18.024884 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-cb2e661a-eb5d-49bd-8a72-4b60471feedb None None] MFA Rules not processed for user `30963f2f464e4f54a8aea0175370a235`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:21:18.026146 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-cb2e661a-eb5d-49bd-8a72-4b60471feedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:21:18.035609 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 751/1520] 10.222.0.50 () {58 vars in 1057 bytes} [Fri May 27 03:21:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:21:18.043231 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b50563-3afa-4fcf-a4fa-53560b302372 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:21:18.059431 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b50563-3afa-4fcf-a4fa-53560b302372 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:21:18.065318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38ebeaa-3cee-4637-893a-7cf3db56c408 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30963f2f464e4f54a8aea0175370a235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:21:18.065961 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ebeaa-3cee-4637-893a-7cf3db56c408 service placement] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:21:18.066238 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ebeaa-3cee-4637-893a-7cf3db56c408 service placement] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:21:18.066514 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38ebeaa-3cee-4637-893a-7cf3db56c408 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:21:18.075760 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38ebeaa-3cee-4637-893a-7cf3db56c408 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:21:18.077443 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38ebeaa-3cee-4637-893a-7cf3db56c408 service placement] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:21:18.088284 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 770/1521] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:21:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:21:20.820766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27033b6d-ce86-4fbb-9921-f1b61b531efa None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:21:20.821414 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27033b6d-ce86-4fbb-9921-f1b61b531efa None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:21:20.821414 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27033b6d-ce86-4fbb-9921-f1b61b531efa None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:21:20.822724 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 752/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:21:20.828066 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005b3f5-e0e4-458e-964e-1c8267953f18 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:21:20.828642 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005b3f5-e0e4-458e-964e-1c8267953f18 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:21:20.828642 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005b3f5-e0e4-458e-964e-1c8267953f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:21:20.847389 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-a005b3f5-e0e4-458e-964e-1c8267953f18 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:21:20.848607 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-a005b3f5-e0e4-458e-964e-1c8267953f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:21:20.856873 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 771/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:21:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:21:21.013382 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27033b6d-ce86-4fbb-9921-f1b61b531efa None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:21:21.030410 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27033b6d-ce86-4fbb-9921-f1b61b531efa None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:21:21.035958 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1c683-696d-4c02-a0d2-90a510f77497 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:21:21.036602 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1c683-696d-4c02-a0d2-90a510f77497 service nova] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:21:21.036882 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1c683-696d-4c02-a0d2-90a510f77497 service nova] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:21:21.037187 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1c683-696d-4c02-a0d2-90a510f77497 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:21:21.046282 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1c683-696d-4c02-a0d2-90a510f77497 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:21:21.047930 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1c683-696d-4c02-a0d2-90a510f77497 service nova] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:21:21.058737 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 753/1524] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:21:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:21:21.977030 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162ce89-e783-49df-9087-1c14ad6d4ed5 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:21:21.977662 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162ce89-e783-49df-9087-1c14ad6d4ed5 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:21:21.977662 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162ce89-e783-49df-9087-1c14ad6d4ed5 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:21:21.978975 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 772/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 03:21:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:21:21.984306 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f23c8d9-375e-40d2-9096-655d7369c793 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:21:21.984801 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f23c8d9-375e-40d2-9096-655d7369c793 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:21:21.984801 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f23c8d9-375e-40d2-9096-655d7369c793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:21:22.004718 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-3f23c8d9-375e-40d2-9096-655d7369c793 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:21:22.005974 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-3f23c8d9-375e-40d2-9096-655d7369c793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:21:22.015193 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 754/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 03:21:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:21:22.166483 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0162ce89-e783-49df-9087-1c14ad6d4ed5 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:21:22.183827 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0162ce89-e783-49df-9087-1c14ad6d4ed5 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:21:22.190044 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667bcd17-09a4-4408-ab22-756d0316f091 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:21:22.190693 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667bcd17-09a4-4408-ab22-756d0316f091 service nova] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:21:22.190971 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667bcd17-09a4-4408-ab22-756d0316f091 service nova] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:21:22.191238 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667bcd17-09a4-4408-ab22-756d0316f091 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:21:22.200157 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667bcd17-09a4-4408-ab22-756d0316f091 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:21:22.201792 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667bcd17-09a4-4408-ab22-756d0316f091 service nova] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:21:22.211601 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 773/1527] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:21:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:21:26.718327 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292dc25e-4bc4-4ec1-b3e7-29df3473638f None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:21:26.719331 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292dc25e-4bc4-4ec1-b3e7-29df3473638f None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:21:26.719331 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292dc25e-4bc4-4ec1-b3e7-29df3473638f None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:21:26.721542 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 755/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:21:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:21:26.730455 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79e153-6e93-45e4-ae89-82ea6e0e1be5 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:21:26.730925 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79e153-6e93-45e4-ae89-82ea6e0e1be5 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:21:26.731028 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79e153-6e93-45e4-ae89-82ea6e0e1be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:21:26.751770 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-7e79e153-6e93-45e4-ae89-82ea6e0e1be5 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:21:26.753000 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-7e79e153-6e93-45e4-ae89-82ea6e0e1be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:21:26.761703 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 774/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:21:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:21:26.800209 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292dc25e-4bc4-4ec1-b3e7-29df3473638f None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:21:26.802965 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-292dc25e-4bc4-4ec1-b3e7-29df3473638f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:21:26.877326 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292dc25e-4bc4-4ec1-b3e7-29df3473638f None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:21:26.883143 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5034c661-c411-415d-9b77-654f7581dc65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:21:26.883832 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034c661-c411-415d-9b77-654f7581dc65 service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:21:26.884115 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034c661-c411-415d-9b77-654f7581dc65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:21:26.884387 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5034c661-c411-415d-9b77-654f7581dc65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:21:26.895294 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5034c661-c411-415d-9b77-654f7581dc65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:21:26.896752 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5034c661-c411-415d-9b77-654f7581dc65 service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:21:26.906747 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 756/1530] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:21:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:21:29.101727 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29543ea2-7db3-4051-8e47-f32f12230ad4 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:21:29.102304 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29543ea2-7db3-4051-8e47-f32f12230ad4 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:21:29.102304 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29543ea2-7db3-4051-8e47-f32f12230ad4 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:21:29.104389 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 775/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:21:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:21:29.110614 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff897e-6a02-4b88-a921-9b137dd8af20 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:21:29.111150 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff897e-6a02-4b88-a921-9b137dd8af20 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:21:29.111150 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff897e-6a02-4b88-a921-9b137dd8af20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:21:29.141423 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-45ff897e-6a02-4b88-a921-9b137dd8af20 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:21:29.143284 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-45ff897e-6a02-4b88-a921-9b137dd8af20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:21:29.155031 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 757/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:21:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:21:29.200098 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29543ea2-7db3-4051-8e47-f32f12230ad4 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:21:29.213135 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29543ea2-7db3-4051-8e47-f32f12230ad4 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:21:29.217728 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce7180b-232a-467f-bfd5-e4ebd46de92e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:21:29.218396 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7180b-232a-467f-bfd5-e4ebd46de92e service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:21:29.218784 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7180b-232a-467f-bfd5-e4ebd46de92e service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:21:29.219053 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7180b-232a-467f-bfd5-e4ebd46de92e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:21:29.230215 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce7180b-232a-467f-bfd5-e4ebd46de92e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:21:29.231696 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce7180b-232a-467f-bfd5-e4ebd46de92e service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:21:29.242973 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 776/1533] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:21:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:21:32.932594 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09154007-59ed-4ad5-9b54-0e08cd382f3a None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:21:32.933572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09154007-59ed-4ad5-9b54-0e08cd382f3a None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:21:32.933572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09154007-59ed-4ad5-9b54-0e08cd382f3a None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:21:32.935792 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 758/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:21:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:21:32.942322 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec9532-98cd-4c92-b347-5a229faac45b None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:21:32.942802 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec9532-98cd-4c92-b347-5a229faac45b None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:21:32.942901 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec9532-98cd-4c92-b347-5a229faac45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:21:32.966985 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-adec9532-98cd-4c92-b347-5a229faac45b None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:21:32.968222 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-adec9532-98cd-4c92-b347-5a229faac45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:21:32.977010 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 777/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:21:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:21:33.252058 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09154007-59ed-4ad5-9b54-0e08cd382f3a None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:21:33.254300 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-09154007-59ed-4ad5-9b54-0e08cd382f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:21:33.316217 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09154007-59ed-4ad5-9b54-0e08cd382f3a None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:21:33.319958 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b514f-cc4f-4f7d-a152-d0c81fdd2363 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:21:33.320603 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b514f-cc4f-4f7d-a152-d0c81fdd2363 service cinder] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:21:33.321168 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b514f-cc4f-4f7d-a152-d0c81fdd2363 service cinder] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:21:33.321443 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b514f-cc4f-4f7d-a152-d0c81fdd2363 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:21:33.331141 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b514f-cc4f-4f7d-a152-d0c81fdd2363 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:21:33.332577 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b514f-cc4f-4f7d-a152-d0c81fdd2363 service cinder] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:21:33.337543 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:21:33.343254 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 759/1536] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:21:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:21:35.585712 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d34c8b-eae5-47a0-ba6d-6f06633a77fc None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:21:35.586832 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d34c8b-eae5-47a0-ba6d-6f06633a77fc None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:21:35.586832 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d34c8b-eae5-47a0-ba6d-6f06633a77fc None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:21:35.588573 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 778/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 03:21:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:21:35.595109 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d27770-68cc-4653-9270-08d5c7d88d9a None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:21:35.595589 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d27770-68cc-4653-9270-08d5c7d88d9a None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:21:35.595694 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d27770-68cc-4653-9270-08d5c7d88d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:21:35.624345 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-d4d27770-68cc-4653-9270-08d5c7d88d9a None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:21:35.625607 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-d4d27770-68cc-4653-9270-08d5c7d88d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:21:35.635151 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 760/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 03:21:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:21:35.909472 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d34c8b-eae5-47a0-ba6d-6f06633a77fc None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:21:35.912449 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-51d34c8b-eae5-47a0-ba6d-6f06633a77fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:21:35.924274 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=94 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:21:35.930936 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d34c8b-eae5-47a0-ba6d-6f06633a77fc None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:21:35.934860 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b73b0-35ce-4cc6-b9de-ddc29d428be7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:21:35.935493 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b73b0-35ce-4cc6-b9de-ddc29d428be7 service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:21:35.935780 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b73b0-35ce-4cc6-b9de-ddc29d428be7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:21:35.936113 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b73b0-35ce-4cc6-b9de-ddc29d428be7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:21:35.945467 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629b73b0-35ce-4cc6-b9de-ddc29d428be7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:21:35.946919 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629b73b0-35ce-4cc6-b9de-ddc29d428be7 service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:21:35.956792 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 779/1539] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:21:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:21:45.616432 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=3 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:21:45.952470 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=5 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:08.930322 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110478b7-2a37-4ef7-b1fb-7de39324c327 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:08.931363 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110478b7-2a37-4ef7-b1fb-7de39324c327 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:08.931915 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110478b7-2a37-4ef7-b1fb-7de39324c327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:08.962626 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-110478b7-2a37-4ef7-b1fb-7de39324c327 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:08.964864 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-110478b7-2a37-4ef7-b1fb-7de39324c327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:08.977458 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 761/1540] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:08.989799 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e0d39-cde4-4414-b9e6-df1b660e1dc5 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:08.990399 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e0d39-cde4-4414-b9e6-df1b660e1dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:08.990399 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e0d39-cde4-4414-b9e6-df1b660e1dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:09.025782 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-3a9e0d39-cde4-4414-b9e6-df1b660e1dc5 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:09.026609 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abeb595-e9f4-49e8-989a-cc855f7d2364 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:09.026900 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abeb595-e9f4-49e8-989a-cc855f7d2364 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:09.027181 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abeb595-e9f4-49e8-989a-cc855f7d2364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:09.027302 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-3a9e0d39-cde4-4414-b9e6-df1b660e1dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:09.041811 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 780/1541] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:09.060882 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e0d39-cde4-4414-b9e6-df1b660e1dc5 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:09.062198 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-9abeb595-e9f4-49e8-989a-cc855f7d2364 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:09.063447 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-9abeb595-e9f4-49e8-989a-cc855f7d2364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:09.075708 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e0d39-cde4-4414-b9e6-df1b660e1dc5 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:09.081857 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 762/1542] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:23:09.084097 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee72b4de-4540-4226-ba3c-719df8624e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:09.085423 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee72b4de-4540-4226-ba3c-719df8624e93 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:09.085705 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee72b4de-4540-4226-ba3c-719df8624e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:09.085975 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee72b4de-4540-4226-ba3c-719df8624e93 admin admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:09.086865 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee72b4de-4540-4226-ba3c-719df8624e93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:23:09.087164 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee72b4de-4540-4226-ba3c-719df8624e93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:09.088566 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: WARNING py.warnings [None req-ee72b4de-4540-4226-ba3c-719df8624e93 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 27 03:23:09.088566 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: warnings.warn(msg) May 27 03:23:09.088566 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: May 27 03:23:09.089617 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee72b4de-4540-4226-ba3c-719df8624e93 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:09.106966 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b479782-1d37-4df7-8095-98e93cc6d102 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:09.107316 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b479782-1d37-4df7-8095-98e93cc6d102 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:09.107535 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b479782-1d37-4df7-8095-98e93cc6d102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:09.112216 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 781/1543] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:23:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:23:09.124934 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee72b4de-4540-4226-ba3c-719df8624e93 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:09.139161 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee72b4de-4540-4226-ba3c-719df8624e93 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:09.151586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b1259-733f-4dab-850d-1308e5941172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:09.152505 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b1259-733f-4dab-850d-1308e5941172 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:09.152794 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b1259-733f-4dab-850d-1308e5941172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:09.153000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-5b479782-1d37-4df7-8095-98e93cc6d102 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:09.153122 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b1259-733f-4dab-850d-1308e5941172 admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:09.153803 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96b1259-733f-4dab-850d-1308e5941172 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:09.154233 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-5b479782-1d37-4df7-8095-98e93cc6d102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:09.155908 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96b1259-733f-4dab-850d-1308e5941172 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:09.170074 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 763/1544] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:23:09.171728 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6ef50-1649-4d44-939c-3e79ecf91c7c None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:09.172017 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6ef50-1649-4d44-939c-3e79ecf91c7c None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:09.172592 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6ef50-1649-4d44-939c-3e79ecf91c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:09.208764 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 782/1545] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:23:09 2022] POST /identity/v3/projects => generated 379 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:23:09.210123 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904cf79a-e07d-481b-bcea-938e9d7bb154 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:09.210461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904cf79a-e07d-481b-bcea-938e9d7bb154 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:09.210740 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904cf79a-e07d-481b-bcea-938e9d7bb154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:09.220804 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-1bc6ef50-1649-4d44-939c-3e79ecf91c7c None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:09.222208 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-1bc6ef50-1649-4d44-939c-3e79ecf91c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:09.260801 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-904cf79a-e07d-481b-bcea-938e9d7bb154 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:09.262469 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-904cf79a-e07d-481b-bcea-938e9d7bb154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:09.347813 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 764/1546] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:09.348498 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc6ef50-1649-4d44-939c-3e79ecf91c7c None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:09.366994 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc6ef50-1649-4d44-939c-3e79ecf91c7c None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:09.376414 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab07934-520d-4b75-88f4-379e7a8365d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:09.377374 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab07934-520d-4b75-88f4-379e7a8365d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:09.377652 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab07934-520d-4b75-88f4-379e7a8365d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:09.378270 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab07934-520d-4b75-88f4-379e7a8365d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:09.379273 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab07934-520d-4b75-88f4-379e7a8365d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:23:09.379578 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab07934-520d-4b75-88f4-379e7a8365d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:09.382359 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab07934-520d-4b75-88f4-379e7a8365d7 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:09.398346 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 765/1547] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:23:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:23:09.399301 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab07934-520d-4b75-88f4-379e7a8365d7 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:09.403470 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 783/1548] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:09.409913 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542b706-372c-4577-9ece-4c955eb7d782 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:09.410196 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542b706-372c-4577-9ece-4c955eb7d782 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:09.410469 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542b706-372c-4577-9ece-4c955eb7d782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:09.423694 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab07934-520d-4b75-88f4-379e7a8365d7 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:09.430054 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cd717d-8d44-4467-b478-f82972588d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:09.431377 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd717d-8d44-4467-b478-f82972588d7f admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:09.432235 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd717d-8d44-4467-b478-f82972588d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:09.432506 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cd717d-8d44-4467-b478-f82972588d7f admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:09.433230 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd717d-8d44-4467-b478-f82972588d7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:09.435408 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cd717d-8d44-4467-b478-f82972588d7f admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:09.445266 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-4542b706-372c-4577-9ece-4c955eb7d782 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:09.446500 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-4542b706-372c-4577-9ece-4c955eb7d782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:09.456453 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 784/1549] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:09.462269 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b2fbef-a43c-4ce8-9ef7-d1d6621c987f None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:09.462269 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b2fbef-a43c-4ce8-9ef7-d1d6621c987f None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:09.462269 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b2fbef-a43c-4ce8-9ef7-d1d6621c987f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:09.462597 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 766/1550] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:23:09 2022] POST /identity/v3/users => generated 290 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:09.462597 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30edc0-3162-49c9-b480-090015505792 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:09.462597 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30edc0-3162-49c9-b480-090015505792 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:09.462597 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad30edc0-3162-49c9-b480-090015505792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:09.479930 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-ad30edc0-3162-49c9-b480-090015505792 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:09.480534 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-d0b2fbef-a43c-4ce8-9ef7-d1d6621c987f None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:09.481167 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-ad30edc0-3162-49c9-b480-090015505792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:09.481733 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-d0b2fbef-a43c-4ce8-9ef7-d1d6621c987f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:09.488869 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 767/1551] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:09.489842 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad30edc0-3162-49c9-b480-090015505792 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:09.489961 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 785/1552] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:09.493821 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015a40a-df37-4795-9636-9af50716dc54 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:09.493821 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015a40a-df37-4795-9636-9af50716dc54 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:09.493821 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b015a40a-df37-4795-9636-9af50716dc54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:09.500042 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad30edc0-3162-49c9-b480-090015505792 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:09.504697 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9ba2b4-8e11-4965-9dc8-d3e4df190cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:09.505395 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ba2b4-8e11-4965-9dc8-d3e4df190cf4 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:09.505673 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ba2b4-8e11-4965-9dc8-d3e4df190cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:09.505943 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9ba2b4-8e11-4965-9dc8-d3e4df190cf4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:09.506625 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9ba2b4-8e11-4965-9dc8-d3e4df190cf4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:09.507675 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: WARNING py.warnings [None req-1d9ba2b4-8e11-4965-9dc8-d3e4df190cf4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:23:09.507675 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: warnings.warn(msg) May 27 03:23:09.507675 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: May 27 03:23:09.508946 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9ba2b4-8e11-4965-9dc8-d3e4df190cf4 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:09.514634 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-b015a40a-df37-4795-9636-9af50716dc54 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:09.515832 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-b015a40a-df37-4795-9636-9af50716dc54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:09.524664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 786/1553] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:09.525799 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a11aac-1040-4337-b6c0-0e1f1f79c037 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:09.526034 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a11aac-1040-4337-b6c0-0e1f1f79c037 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:09.526510 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a11aac-1040-4337-b6c0-0e1f1f79c037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:09.542303 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 768/1554] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:23:09 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:09.542303 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472965a2-cdfa-4920-b1c8-8b7330cc646c None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:09.542303 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472965a2-cdfa-4920-b1c8-8b7330cc646c None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:09.542303 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472965a2-cdfa-4920-b1c8-8b7330cc646c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:09.549425 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-01a11aac-1040-4337-b6c0-0e1f1f79c037 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:09.550636 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-01a11aac-1040-4337-b6c0-0e1f1f79c037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:09.558867 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-472965a2-cdfa-4920-b1c8-8b7330cc646c None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:09.560021 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-472965a2-cdfa-4920-b1c8-8b7330cc646c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:09.612158 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 787/1555] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:09.613043 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a11aac-1040-4337-b6c0-0e1f1f79c037 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:09.622173 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 769/1556] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:09.623301 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a11aac-1040-4337-b6c0-0e1f1f79c037 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:09.623966 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efde09-a7e8-4961-aada-b86748d69561 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:09.624458 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efde09-a7e8-4961-aada-b86748d69561 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:09.624739 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efde09-a7e8-4961-aada-b86748d69561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:09.633213 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5300a57-b03d-44ef-a608-d7b0d605310d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:09.634178 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5300a57-b03d-44ef-a608-d7b0d605310d admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:09.634446 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5300a57-b03d-44ef-a608-d7b0d605310d admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:09.635607 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5300a57-b03d-44ef-a608-d7b0d605310d admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:09.636050 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5300a57-b03d-44ef-a608-d7b0d605310d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:09.637778 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5300a57-b03d-44ef-a608-d7b0d605310d admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:09.645252 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-64efde09-a7e8-4961-aada-b86748d69561 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:09.646460 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-64efde09-a7e8-4961-aada-b86748d69561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:09.649360 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 788/1557] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:09.650027 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5300a57-b03d-44ef-a608-d7b0d605310d admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:09.656193 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 770/1558] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:09.657483 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4fb03b-ad70-45c0-9ef3-a9783e192d5e None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:09.658303 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4fb03b-ad70-45c0-9ef3-a9783e192d5e None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:09.658303 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4fb03b-ad70-45c0-9ef3-a9783e192d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:09.662307 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5300a57-b03d-44ef-a608-d7b0d605310d admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:09.668710 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c34f8-ef6f-4169-9dfc-8965ece8b53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:09.669420 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c34f8-ef6f-4169-9dfc-8965ece8b53e admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:09.669828 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c34f8-ef6f-4169-9dfc-8965ece8b53e admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:09.670102 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c34f8-ef6f-4169-9dfc-8965ece8b53e admin admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:09.670831 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7c34f8-ef6f-4169-9dfc-8965ece8b53e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:23:09.671276 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7c34f8-ef6f-4169-9dfc-8965ece8b53e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:09.672909 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7c34f8-ef6f-4169-9dfc-8965ece8b53e admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:09.677274 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-eb4fb03b-ad70-45c0-9ef3-a9783e192d5e None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:09.678473 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-eb4fb03b-ad70-45c0-9ef3-a9783e192d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:09.690267 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 771/1559] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:09.690465 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 789/1560] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:23:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:23:09.690569 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4fb03b-ad70-45c0-9ef3-a9783e192d5e None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:09.690661 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c34f8-ef6f-4169-9dfc-8965ece8b53e admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:09.699915 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c34f8-ef6f-4169-9dfc-8965ece8b53e admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:09.700111 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4fb03b-ad70-45c0-9ef3-a9783e192d5e None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:09.704710 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844ce79f-9b67-495e-9495-07e35fec172d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:09.704863 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd0f767-14e5-449b-af82-11b859615ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:09.705416 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ce79f-9b67-495e-9495-07e35fec172d admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:09.705529 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0f767-14e5-449b-af82-11b859615ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:09.705695 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ce79f-9b67-495e-9495-07e35fec172d admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:09.705804 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0f767-14e5-449b-af82-11b859615ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:09.705966 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844ce79f-9b67-495e-9495-07e35fec172d admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:09.706076 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd0f767-14e5-449b-af82-11b859615ca7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:09.706650 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ce79f-9b67-495e-9495-07e35fec172d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:09.706766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd0f767-14e5-449b-af82-11b859615ca7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:23:09.707076 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd0f767-14e5-449b-af82-11b859615ca7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:09.707678 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: WARNING py.warnings [None req-844ce79f-9b67-495e-9495-07e35fec172d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:23:09.707678 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: warnings.warn(msg) May 27 03:23:09.707678 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: May 27 03:23:09.708702 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd0f767-14e5-449b-af82-11b859615ca7 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:09.708998 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844ce79f-9b67-495e-9495-07e35fec172d admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:09.711312 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:09.722878 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 772/1561] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:23:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:23:09.724025 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d327ca-2006-4146-9803-ab124d61c44e None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:09.724318 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d327ca-2006-4146-9803-ab124d61c44e None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:09.724628 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d327ca-2006-4146-9803-ab124d61c44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:09.733422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:09.742921 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 790/1562] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:23:09 2022] POST /identity/v3/users => generated 286 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:09.742921 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936ff81-6047-4e94-8cae-5a364e8d0553 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:09.742921 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936ff81-6047-4e94-8cae-5a364e8d0553 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:09.742921 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c936ff81-6047-4e94-8cae-5a364e8d0553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:09.744493 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-c9d327ca-2006-4146-9803-ab124d61c44e None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:09.745902 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-c9d327ca-2006-4146-9803-ab124d61c44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:09.753734 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 773/1563] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:09.754569 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d327ca-2006-4146-9803-ab124d61c44e None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:09.761721 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-c936ff81-6047-4e94-8cae-5a364e8d0553 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:09.762919 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-c936ff81-6047-4e94-8cae-5a364e8d0553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:09.764166 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d327ca-2006-4146-9803-ab124d61c44e None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:09.769284 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523dbce-dfcf-4d5b-a439-114aca606b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:09.770052 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523dbce-dfcf-4d5b-a439-114aca606b2b admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:09.770432 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523dbce-dfcf-4d5b-a439-114aca606b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:09.770803 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523dbce-dfcf-4d5b-a439-114aca606b2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:09.771029 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 791/1564] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:09.771707 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523dbce-dfcf-4d5b-a439-114aca606b2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:09.771826 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936ff81-6047-4e94-8cae-5a364e8d0553 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:09.773617 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523dbce-dfcf-4d5b-a439-114aca606b2b admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:09.781216 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c936ff81-6047-4e94-8cae-5a364e8d0553 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:09.784986 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 774/1565] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:09.785790 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523dbce-dfcf-4d5b-a439-114aca606b2b admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:09.788628 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36358d6d-61d9-4c09-bfd2-5db2edc677a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:09.789313 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36358d6d-61d9-4c09-bfd2-5db2edc677a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:09.789596 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36358d6d-61d9-4c09-bfd2-5db2edc677a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:09.789864 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36358d6d-61d9-4c09-bfd2-5db2edc677a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:09.790585 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36358d6d-61d9-4c09-bfd2-5db2edc677a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:23:09.791219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36358d6d-61d9-4c09-bfd2-5db2edc677a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:09.793271 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36358d6d-61d9-4c09-bfd2-5db2edc677a1 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:09.797120 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523dbce-dfcf-4d5b-a439-114aca606b2b admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:09.802622 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ab021-8806-46fc-8b1f-21c151a60f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:09.803469 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ab021-8806-46fc-8b1f-21c151a60f55 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:09.803750 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ab021-8806-46fc-8b1f-21c151a60f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:09.804022 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ab021-8806-46fc-8b1f-21c151a60f55 admin admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:09.804731 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300ab021-8806-46fc-8b1f-21c151a60f55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:23:09.805178 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 792/1566] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:23:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:23:09.805306 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300ab021-8806-46fc-8b1f-21c151a60f55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:09.805888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36358d6d-61d9-4c09-bfd2-5db2edc677a1 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:09.806681 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300ab021-8806-46fc-8b1f-21c151a60f55 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:09.816840 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36358d6d-61d9-4c09-bfd2-5db2edc677a1 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:09.819903 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 775/1567] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:23:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:23:09.820859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ab021-8806-46fc-8b1f-21c151a60f55 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:09.821382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f8516-7f77-4885-a8dd-84d68513c51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:09.822030 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f8516-7f77-4885-a8dd-84d68513c51f admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:09.822414 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f8516-7f77-4885-a8dd-84d68513c51f admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:09.822880 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f8516-7f77-4885-a8dd-84d68513c51f admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:09.823777 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f8516-7f77-4885-a8dd-84d68513c51f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:09.825726 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f8516-7f77-4885-a8dd-84d68513c51f admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:09.830521 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ab021-8806-46fc-8b1f-21c151a60f55 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:09.835370 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd3b1d0-ac32-4b23-af87-4241551cd720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:09.836029 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3b1d0-ac32-4b23-af87-4241551cd720 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:09.836307 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3b1d0-ac32-4b23-af87-4241551cd720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:09.836579 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd3b1d0-ac32-4b23-af87-4241551cd720 admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:09.837299 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd3b1d0-ac32-4b23-af87-4241551cd720 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:09.839625 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd3b1d0-ac32-4b23-af87-4241551cd720 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:09.857154 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 793/1568] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:23:09 2022] POST /identity/v3/projects => generated 391 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:09.857835 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f8516-7f77-4885-a8dd-84d68513c51f admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:09.870624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 776/1569] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:23:09 2022] POST /identity/v3/projects => generated 389 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:09.871329 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd3b1d0-ac32-4b23-af87-4241551cd720 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:09.924596 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f8516-7f77-4885-a8dd-84d68513c51f admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:09.942484 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd3b1d0-ac32-4b23-af87-4241551cd720 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:09.950917 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6656bf0d-ba33-426a-9d37-834f6aed2df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:09.951964 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6656bf0d-ba33-426a-9d37-834f6aed2df8 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:09.952414 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6656bf0d-ba33-426a-9d37-834f6aed2df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:09.952852 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6656bf0d-ba33-426a-9d37-834f6aed2df8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:09.954049 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6656bf0d-ba33-426a-9d37-834f6aed2df8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:23:09.954565 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6656bf0d-ba33-426a-9d37-834f6aed2df8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:09.957262 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6656bf0d-ba33-426a-9d37-834f6aed2df8 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:09.976811 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 777/1570] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:23:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:23:09.978030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6656bf0d-ba33-426a-9d37-834f6aed2df8 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:09.990688 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6656bf0d-ba33-426a-9d37-834f6aed2df8 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:09.996827 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7f02b5-134d-4b66-a68d-ff1beca2f606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:09.997857 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7f02b5-134d-4b66-a68d-ff1beca2f606 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:09.998247 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7f02b5-134d-4b66-a68d-ff1beca2f606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:09.998568 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7f02b5-134d-4b66-a68d-ff1beca2f606 admin admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:09.999352 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7f02b5-134d-4b66-a68d-ff1beca2f606 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:23:09.999877 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7f02b5-134d-4b66-a68d-ff1beca2f606 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:10.001654 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7f02b5-134d-4b66-a68d-ff1beca2f606 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:10.009831 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38254459-39ad-4b24-b2b0-2464ab94b819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:10.010564 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38254459-39ad-4b24-b2b0-2464ab94b819 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:10.010838 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38254459-39ad-4b24-b2b0-2464ab94b819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:10.011106 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38254459-39ad-4b24-b2b0-2464ab94b819 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:10.011857 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38254459-39ad-4b24-b2b0-2464ab94b819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:10.013494 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38254459-39ad-4b24-b2b0-2464ab94b819 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:10.024129 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 778/1571] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:23:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:23:10.024129 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7f02b5-134d-4b66-a68d-ff1beca2f606 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:10.030079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 794/1572] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:09 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:10.031224 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38254459-39ad-4b24-b2b0-2464ab94b819 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:10.036690 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7f02b5-134d-4b66-a68d-ff1beca2f606 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:10.040859 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38254459-39ad-4b24-b2b0-2464ab94b819 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:10.041732 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96103204-c64d-4c02-83c8-fc4a957440a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:10.042746 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96103204-c64d-4c02-83c8-fc4a957440a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:10.043035 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96103204-c64d-4c02-83c8-fc4a957440a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:10.043370 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96103204-c64d-4c02-83c8-fc4a957440a8 admin admin] PATH_INFO: `/v3/projects/a0a3ae93f6ab44af93b0538061809b98/users/1b22fa493ae04d5fac13b94cff154eee/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:10.045611 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef5e21-d087-4523-acbc-f1ac36e70289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:10.046273 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef5e21-d087-4523-acbc-f1ac36e70289 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:10.046554 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef5e21-d087-4523-acbc-f1ac36e70289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:10.046830 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ef5e21-d087-4523-acbc-f1ac36e70289 admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:10.047513 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef5e21-d087-4523-acbc-f1ac36e70289 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:10.049528 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ef5e21-d087-4523-acbc-f1ac36e70289 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:10.062022 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96103204-c64d-4c02-83c8-fc4a957440a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0a3ae93f6ab44af93b0538061809b98, user_id=1b22fa493ae04d5fac13b94cff154eee, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:10.063072 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: WARNING py.warnings [None req-96103204-c64d-4c02-83c8-fc4a957440a8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:23:10.063072 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: warnings.warn(msg) May 27 03:23:10.063072 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: May 27 03:23:10.065094 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96103204-c64d-4c02-83c8-fc4a957440a8 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:10.079023 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 795/1573] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:23:10 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:10.079756 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 779/1574] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:23:10 2022] PUT /identity/v3/projects/a0a3ae93f6ab44af93b0538061809b98/users/1b22fa493ae04d5fac13b94cff154eee/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:10.079943 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef5e21-d087-4523-acbc-f1ac36e70289 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:10.080535 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96103204-c64d-4c02-83c8-fc4a957440a8 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:10.128879 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:10.151017 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96103204-c64d-4c02-83c8-fc4a957440a8 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:10.152187 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ef5e21-d087-4523-acbc-f1ac36e70289 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:10.156072 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725dafff-f350-48c9-8007-99dee5341e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:10.156813 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725dafff-f350-48c9-8007-99dee5341e93 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:10.157006 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eab9d6-b483-438e-ad6d-d10cd7b0ce05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:10.157210 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725dafff-f350-48c9-8007-99dee5341e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:10.157485 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725dafff-f350-48c9-8007-99dee5341e93 admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:10.157633 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eab9d6-b483-438e-ad6d-d10cd7b0ce05 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:10.157912 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eab9d6-b483-438e-ad6d-d10cd7b0ce05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:10.158166 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725dafff-f350-48c9-8007-99dee5341e93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:10.158279 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eab9d6-b483-438e-ad6d-d10cd7b0ce05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:10.158879 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eab9d6-b483-438e-ad6d-d10cd7b0ce05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:10.160080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725dafff-f350-48c9-8007-99dee5341e93 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:10.160781 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eab9d6-b483-438e-ad6d-d10cd7b0ce05 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:10.166878 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:10.185571 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 780/1575] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:23:10 2022] POST /identity/v3/users => generated 296 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:10.186364 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725dafff-f350-48c9-8007-99dee5341e93 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:10.186730 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 796/1576] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:23:10 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:10.187426 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eab9d6-b483-438e-ad6d-d10cd7b0ce05 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:10.198537 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725dafff-f350-48c9-8007-99dee5341e93 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:10.248630 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eab9d6-b483-438e-ad6d-d10cd7b0ce05 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:10.253102 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f491de2e-a683-49a5-9826-bf5fc8e9ddb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:10.254045 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f491de2e-a683-49a5-9826-bf5fc8e9ddb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:10.254189 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f491de2e-a683-49a5-9826-bf5fc8e9ddb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:10.254360 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f491de2e-a683-49a5-9826-bf5fc8e9ddb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:10.255050 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f491de2e-a683-49a5-9826-bf5fc8e9ddb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:10.255881 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528084cf-cdfa-477f-a54f-959c2d594490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:10.256565 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528084cf-cdfa-477f-a54f-959c2d594490 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:10.256852 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528084cf-cdfa-477f-a54f-959c2d594490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:10.257031 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f491de2e-a683-49a5-9826-bf5fc8e9ddb2 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:10.257148 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528084cf-cdfa-477f-a54f-959c2d594490 admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:10.257831 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528084cf-cdfa-477f-a54f-959c2d594490 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:10.259887 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528084cf-cdfa-477f-a54f-959c2d594490 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:10.286734 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 781/1577] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:23:10 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:10.290289 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 797/1578] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:23:10 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:10.290289 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f491de2e-a683-49a5-9826-bf5fc8e9ddb2 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:10.290529 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528084cf-cdfa-477f-a54f-959c2d594490 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:10.301472 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528084cf-cdfa-477f-a54f-959c2d594490 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:10.356434 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f491de2e-a683-49a5-9826-bf5fc8e9ddb2 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:10.357940 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80058a9b-4f3a-459f-b9ae-f30f6eca6b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:10.358612 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80058a9b-4f3a-459f-b9ae-f30f6eca6b73 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:10.358891 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80058a9b-4f3a-459f-b9ae-f30f6eca6b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:10.359169 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80058a9b-4f3a-459f-b9ae-f30f6eca6b73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:10.359866 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80058a9b-4f3a-459f-b9ae-f30f6eca6b73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:10.361219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4641d87-8d12-4fae-ae5b-5d6eaf209bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:10.361892 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80058a9b-4f3a-459f-b9ae-f30f6eca6b73 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:10.362008 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4641d87-8d12-4fae-ae5b-5d6eaf209bcb admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:10.362173 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4641d87-8d12-4fae-ae5b-5d6eaf209bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:10.362477 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4641d87-8d12-4fae-ae5b-5d6eaf209bcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:10.363211 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4641d87-8d12-4fae-ae5b-5d6eaf209bcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:10.364829 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4641d87-8d12-4fae-ae5b-5d6eaf209bcb admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:10.375031 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 798/1579] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:10.375819 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4641d87-8d12-4fae-ae5b-5d6eaf209bcb admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:10.385652 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4641d87-8d12-4fae-ae5b-5d6eaf209bcb admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:10.390213 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c385a74d-89bb-4cec-88d4-c99d5233ed92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:10.390874 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c385a74d-89bb-4cec-88d4-c99d5233ed92 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:10.391156 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c385a74d-89bb-4cec-88d4-c99d5233ed92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:10.391881 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c385a74d-89bb-4cec-88d4-c99d5233ed92 admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:10.392065 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 782/1580] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:23:10 2022] POST /identity/v3/projects => generated 405 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:10.392565 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c385a74d-89bb-4cec-88d4-c99d5233ed92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:10.393395 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7073bb9-8983-44f6-a6bc-0b8d4a2d06b9 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:10.393674 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7073bb9-8983-44f6-a6bc-0b8d4a2d06b9 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:10.393949 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7073bb9-8983-44f6-a6bc-0b8d4a2d06b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:10.394613 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c385a74d-89bb-4cec-88d4-c99d5233ed92 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:10.415283 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-e7073bb9-8983-44f6-a6bc-0b8d4a2d06b9 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:10.416660 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-e7073bb9-8983-44f6-a6bc-0b8d4a2d06b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:10.418316 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 799/1581] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:23:10 2022] POST /identity/v3/users => generated 293 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:10.419138 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c385a74d-89bb-4cec-88d4-c99d5233ed92 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:10.430229 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c385a74d-89bb-4cec-88d4-c99d5233ed92 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:10.457579 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:10.482986 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 783/1582] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:10.483816 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7073bb9-8983-44f6-a6bc-0b8d4a2d06b9 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:10.489389 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:10.492509 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb047f0-b9ce-42cb-bb00-0e74c2c1bf9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:10.493432 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb047f0-b9ce-42cb-bb00-0e74c2c1bf9b admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:10.493575 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7073bb9-8983-44f6-a6bc-0b8d4a2d06b9 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:10.493677 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb047f0-b9ce-42cb-bb00-0e74c2c1bf9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:10.493857 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb047f0-b9ce-42cb-bb00-0e74c2c1bf9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:10.494830 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb047f0-b9ce-42cb-bb00-0e74c2c1bf9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:10.496539 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb047f0-b9ce-42cb-bb00-0e74c2c1bf9b admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:10.499445 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279af4a9-59b9-46bd-b7b8-ea2d13fb9b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:10.500423 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279af4a9-59b9-46bd-b7b8-ea2d13fb9b57 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:10.501037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279af4a9-59b9-46bd-b7b8-ea2d13fb9b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:10.501320 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279af4a9-59b9-46bd-b7b8-ea2d13fb9b57 admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:10.502021 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279af4a9-59b9-46bd-b7b8-ea2d13fb9b57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:10.504010 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279af4a9-59b9-46bd-b7b8-ea2d13fb9b57 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:10.507517 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 800/1583] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:10.508318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb047f0-b9ce-42cb-bb00-0e74c2c1bf9b admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:10.520141 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb047f0-b9ce-42cb-bb00-0e74c2c1bf9b admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:10.524915 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207423f-8303-4533-a703-74f180ce3327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:10.526119 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207423f-8303-4533-a703-74f180ce3327 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:10.526588 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 784/1584] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:23:10 2022] POST /identity/v3/users => generated 292 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:10.526724 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207423f-8303-4533-a703-74f180ce3327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:10.526840 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207423f-8303-4533-a703-74f180ce3327 admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:10.527403 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279af4a9-59b9-46bd-b7b8-ea2d13fb9b57 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:10.527581 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0207423f-8303-4533-a703-74f180ce3327 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:10.529619 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0207423f-8303-4533-a703-74f180ce3327 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:10.537463 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279af4a9-59b9-46bd-b7b8-ea2d13fb9b57 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:10.543740 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8dfee6-23ca-4d36-b7bd-b44c59c7f8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:10.544408 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8dfee6-23ca-4d36-b7bd-b44c59c7f8bd admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:10.544687 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8dfee6-23ca-4d36-b7bd-b44c59c7f8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:10.545186 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8dfee6-23ca-4d36-b7bd-b44c59c7f8bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:10.545715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8dfee6-23ca-4d36-b7bd-b44c59c7f8bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:10.547318 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8dfee6-23ca-4d36-b7bd-b44c59c7f8bd admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:10.549928 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 801/1585] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:23:10 2022] POST /identity/v3/users => generated 289 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:10.553146 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207423f-8303-4533-a703-74f180ce3327 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:10.561812 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 785/1586] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:10.561812 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8dfee6-23ca-4d36-b7bd-b44c59c7f8bd admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:10.562073 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207423f-8303-4533-a703-74f180ce3327 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:10.565563 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5fe3b4-62fd-4ef7-9c89-98caeaaf0737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:10.566449 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5fe3b4-62fd-4ef7-9c89-98caeaaf0737 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:10.566730 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5fe3b4-62fd-4ef7-9c89-98caeaaf0737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:10.567007 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5fe3b4-62fd-4ef7-9c89-98caeaaf0737 admin admin] PATH_INFO: `/v3/projects/b5bae16346a94be89a5af009f5937fd6/users/8054e514b45e42f38a6713ba1d823cec/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:10.569366 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8dfee6-23ca-4d36-b7bd-b44c59c7f8bd admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:10.573945 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef9812b-02de-4bc2-bf37-78dd417e4764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:10.574597 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9812b-02de-4bc2-bf37-78dd417e4764 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:10.574878 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9812b-02de-4bc2-bf37-78dd417e4764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:10.575151 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9812b-02de-4bc2-bf37-78dd417e4764 admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:10.575917 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef9812b-02de-4bc2-bf37-78dd417e4764 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:10.577774 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef9812b-02de-4bc2-bf37-78dd417e4764 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:10.582057 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5fe3b4-62fd-4ef7-9c89-98caeaaf0737 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5bae16346a94be89a5af009f5937fd6, user_id=8054e514b45e42f38a6713ba1d823cec, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:10.584716 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5fe3b4-62fd-4ef7-9c89-98caeaaf0737 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:10.602211 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 786/1587] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:23:10 2022] POST /identity/v3/users => generated 303 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:10.602391 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 802/1588] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:23:10 2022] PUT /identity/v3/projects/b5bae16346a94be89a5af009f5937fd6/users/8054e514b45e42f38a6713ba1d823cec/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:10.602508 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef9812b-02de-4bc2-bf37-78dd417e4764 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:10.606086 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5fe3b4-62fd-4ef7-9c89-98caeaaf0737 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:10.610544 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef9812b-02de-4bc2-bf37-78dd417e4764 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:10.662104 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5fe3b4-62fd-4ef7-9c89-98caeaaf0737 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:10.665674 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b80c9-6680-48f4-ade4-2d0d1cc93d82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:10.666509 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b80c9-6680-48f4-ade4-2d0d1cc93d82 service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:10.666790 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa0d02-4326-46b5-8874-ac119bf274e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:10.666914 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b80c9-6680-48f4-ade4-2d0d1cc93d82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:10.667108 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b80c9-6680-48f4-ade4-2d0d1cc93d82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:10.667491 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa0d02-4326-46b5-8874-ac119bf274e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:10.667751 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa0d02-4326-46b5-8874-ac119bf274e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:10.668030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa0d02-4326-46b5-8874-ac119bf274e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:10.668757 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfa0d02-4326-46b5-8874-ac119bf274e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:10.670409 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfa0d02-4326-46b5-8874-ac119bf274e0 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:10.676371 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3b80c9-6680-48f4-ade4-2d0d1cc93d82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:10.677906 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3b80c9-6680-48f4-ade4-2d0d1cc93d82 service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:10.680277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 787/1589] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:10.681106 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa0d02-4326-46b5-8874-ac119bf274e0 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:10.689484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 803/1590] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:10.690348 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b80c9-6680-48f4-ade4-2d0d1cc93d82 service neutron] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:10.692030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa0d02-4326-46b5-8874-ac119bf274e0 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:10.698189 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7298211-c26c-4cc3-a39e-170b05f025e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:10.698860 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7298211-c26c-4cc3-a39e-170b05f025e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:10.699143 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7298211-c26c-4cc3-a39e-170b05f025e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:10.699416 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7298211-c26c-4cc3-a39e-170b05f025e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:10.699872 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b80c9-6680-48f4-ade4-2d0d1cc93d82 service neutron] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:10.700143 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7298211-c26c-4cc3-a39e-170b05f025e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:10.701805 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7298211-c26c-4cc3-a39e-170b05f025e7 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:10.705282 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af405d0-79ad-4e6f-91da-4965fa5dab15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:10.706053 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af405d0-79ad-4e6f-91da-4965fa5dab15 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:10.706375 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af405d0-79ad-4e6f-91da-4965fa5dab15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:10.706715 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af405d0-79ad-4e6f-91da-4965fa5dab15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:10.707503 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af405d0-79ad-4e6f-91da-4965fa5dab15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:10.709241 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af405d0-79ad-4e6f-91da-4965fa5dab15 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:10.714276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 788/1591] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:10.715834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7298211-c26c-4cc3-a39e-170b05f025e7 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:10.720594 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 804/1592] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:10.721591 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af405d0-79ad-4e6f-91da-4965fa5dab15 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:10.730323 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7298211-c26c-4cc3-a39e-170b05f025e7 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:10.731780 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af405d0-79ad-4e6f-91da-4965fa5dab15 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:10.735364 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5c6206-f9ef-495c-8217-8ccb36eb93a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:10.736150 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c6206-f9ef-495c-8217-8ccb36eb93a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:10.736503 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c6206-f9ef-495c-8217-8ccb36eb93a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:10.736842 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c6206-f9ef-495c-8217-8ccb36eb93a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:10.737666 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887377ca-0e16-4ab9-a25f-88fb7d6a3b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:10.737795 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5c6206-f9ef-495c-8217-8ccb36eb93a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:10.738576 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887377ca-0e16-4ab9-a25f-88fb7d6a3b79 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:10.739012 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887377ca-0e16-4ab9-a25f-88fb7d6a3b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:10.739442 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887377ca-0e16-4ab9-a25f-88fb7d6a3b79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:10.739670 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5c6206-f9ef-495c-8217-8ccb36eb93a8 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:10.740632 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887377ca-0e16-4ab9-a25f-88fb7d6a3b79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:10.742801 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887377ca-0e16-4ab9-a25f-88fb7d6a3b79 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:10.754953 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 789/1593] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:10.756364 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5c6206-f9ef-495c-8217-8ccb36eb93a8 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:10.757120 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 805/1594] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:10.758572 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a4fb6-6e31-48da-b699-99c5110a4dcb None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:10.758940 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a4fb6-6e31-48da-b699-99c5110a4dcb None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:10.759275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a4fb6-6e31-48da-b699-99c5110a4dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:10.772556 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5c6206-f9ef-495c-8217-8ccb36eb93a8 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:10.778561 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d822009-2ab0-484c-99ec-ac7cc93b4388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:10.779067 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d822009-2ab0-484c-99ec-ac7cc93b4388 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:10.779416 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d822009-2ab0-484c-99ec-ac7cc93b4388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:10.779776 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d822009-2ab0-484c-99ec-ac7cc93b4388 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:10.780574 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d822009-2ab0-484c-99ec-ac7cc93b4388 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:10.782527 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d822009-2ab0-484c-99ec-ac7cc93b4388 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:10.791223 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-a85a4fb6-6e31-48da-b699-99c5110a4dcb None None] MFA Rules not processed for user `8054e514b45e42f38a6713ba1d823cec`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:10.792442 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-a85a4fb6-6e31-48da-b699-99c5110a4dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:10.794720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 790/1595] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:10.795719 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d822009-2ab0-484c-99ec-ac7cc93b4388 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:10.807336 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d822009-2ab0-484c-99ec-ac7cc93b4388 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:10.813070 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35246ca1-b3f6-4fe5-ba4c-d9523487162f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:10.813917 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35246ca1-b3f6-4fe5-ba4c-d9523487162f admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:10.814257 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35246ca1-b3f6-4fe5-ba4c-d9523487162f admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:10.814982 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35246ca1-b3f6-4fe5-ba4c-d9523487162f admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:10.815771 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35246ca1-b3f6-4fe5-ba4c-d9523487162f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:10.818359 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35246ca1-b3f6-4fe5-ba4c-d9523487162f admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:10.832092 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 791/1596] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:10.833888 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35246ca1-b3f6-4fe5-ba4c-d9523487162f admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:10.844426 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35246ca1-b3f6-4fe5-ba4c-d9523487162f admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:10.849622 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b91f0-d871-426b-a804-921df8e6b284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:10.850415 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b91f0-d871-426b-a804-921df8e6b284 admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:10.850794 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b91f0-d871-426b-a804-921df8e6b284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:10.851114 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b91f0-d871-426b-a804-921df8e6b284 admin admin] PATH_INFO: `/v3/projects/6db8193dcee944008c7b41889bfa67a3/users/468a46d55bd342739b9cd78bcd3a77fc/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:10.851596 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:10.872791 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2b91f0-d871-426b-a804-921df8e6b284 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6db8193dcee944008c7b41889bfa67a3, user_id=468a46d55bd342739b9cd78bcd3a77fc, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:10.877096 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 806/1597] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:10.877096 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85a4fb6-6e31-48da-b699-99c5110a4dcb None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:10.877346 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2b91f0-d871-426b-a804-921df8e6b284 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:10.889696 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85a4fb6-6e31-48da-b699-99c5110a4dcb None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:10.892973 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 792/1598] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:23:10 2022] PUT /identity/v3/projects/6db8193dcee944008c7b41889bfa67a3/users/468a46d55bd342739b9cd78bcd3a77fc/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:10.893579 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b91f0-d871-426b-a804-921df8e6b284 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:10.955833 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:10.962226 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb2277-b751-4be0-9b47-167e615dab65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:10.963180 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb2277-b751-4be0-9b47-167e615dab65 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:10.963507 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb2277-b751-4be0-9b47-167e615dab65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:10.964008 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb2277-b751-4be0-9b47-167e615dab65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:10.964810 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbb2277-b751-4be0-9b47-167e615dab65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:10.967581 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbb2277-b751-4be0-9b47-167e615dab65 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:10.972130 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b91f0-d871-426b-a804-921df8e6b284 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:10.977190 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3cc3f-5ced-4e14-a2f4-89e65e58be89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:10.977874 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3cc3f-5ced-4e14-a2f4-89e65e58be89 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:10.978186 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3cc3f-5ced-4e14-a2f4-89e65e58be89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:10.978560 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3cc3f-5ced-4e14-a2f4-89e65e58be89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:10.979367 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c3cc3f-5ced-4e14-a2f4-89e65e58be89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:10.981088 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c3cc3f-5ced-4e14-a2f4-89e65e58be89 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:10.981246 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 807/1599] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:10.983250 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb2277-b751-4be0-9b47-167e615dab65 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:10.992352 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 793/1600] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:10.993347 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3cc3f-5ced-4e14-a2f4-89e65e58be89 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:10.995231 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb2277-b751-4be0-9b47-167e615dab65 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:10.999950 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d4b4d1-aed0-4766-b53d-5e412b2dc2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:11.000976 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4b4d1-aed0-4766-b53d-5e412b2dc2a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:11.001283 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4b4d1-aed0-4766-b53d-5e412b2dc2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:11.001556 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4b4d1-aed0-4766-b53d-5e412b2dc2a5 admin admin] PATH_INFO: `/v3/projects/7e2e4b63d6af43058e0dc76d8ceac844/users/dd9938d83dc84ae494310741e2cba2ec/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:11.005960 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c3cc3f-5ced-4e14-a2f4-89e65e58be89 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:11.012013 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebdd168-3285-4354-8fa6-7ad775e16616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:11.012692 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebdd168-3285-4354-8fa6-7ad775e16616 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:11.012995 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebdd168-3285-4354-8fa6-7ad775e16616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:11.014715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebdd168-3285-4354-8fa6-7ad775e16616 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:11.014715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebdd168-3285-4354-8fa6-7ad775e16616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:11.015781 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebdd168-3285-4354-8fa6-7ad775e16616 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:11.018273 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d4b4d1-aed0-4766-b53d-5e412b2dc2a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e2e4b63d6af43058e0dc76d8ceac844, user_id=dd9938d83dc84ae494310741e2cba2ec, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:11.021067 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d4b4d1-aed0-4766-b53d-5e412b2dc2a5 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:11.029846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 794/1601] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:11.030562 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebdd168-3285-4354-8fa6-7ad775e16616 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:11.041131 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 808/1602] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:23:10 2022] PUT /identity/v3/projects/7e2e4b63d6af43058e0dc76d8ceac844/users/dd9938d83dc84ae494310741e2cba2ec/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:11.045171 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d4b4d1-aed0-4766-b53d-5e412b2dc2a5 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:11.055458 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebdd168-3285-4354-8fa6-7ad775e16616 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:11.130938 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd14f87-1bcc-4141-a8df-3201acf28ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:11.133029 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd14f87-1bcc-4141-a8df-3201acf28ac1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:11.133623 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd14f87-1bcc-4141-a8df-3201acf28ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:11.134119 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd14f87-1bcc-4141-a8df-3201acf28ac1 admin admin] PATH_INFO: `/v3/projects/d1f5e434c7534753aea9c96091527cd1/users/0731eb80ac2f4eae82a8aee6454c3840/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:11.138799 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d4b4d1-aed0-4766-b53d-5e412b2dc2a5 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:11.144824 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb85d4-1c02-4284-82c5-fa6b6bf55fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:11.145757 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb85d4-1c02-4284-82c5-fa6b6bf55fb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:11.146041 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb85d4-1c02-4284-82c5-fa6b6bf55fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:11.146860 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb85d4-1c02-4284-82c5-fa6b6bf55fb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:11.147092 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cb85d4-1c02-4284-82c5-fa6b6bf55fb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:11.149592 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cb85d4-1c02-4284-82c5-fa6b6bf55fb9 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:11.157483 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd14f87-1bcc-4141-a8df-3201acf28ac1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1f5e434c7534753aea9c96091527cd1, user_id=0731eb80ac2f4eae82a8aee6454c3840, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:11.160913 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd14f87-1bcc-4141-a8df-3201acf28ac1 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:11.183687 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 809/1603] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:23:11 2022] POST /identity/v3/projects => generated 371 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:11.185079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2f1a6-5c4f-4b90-9ea6-d940d2bd5f5e None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:11.185375 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2f1a6-5c4f-4b90-9ea6-d940d2bd5f5e None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:11.185680 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2f1a6-5c4f-4b90-9ea6-d940d2bd5f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:11.197133 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 795/1604] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:23:11 2022] PUT /identity/v3/projects/d1f5e434c7534753aea9c96091527cd1/users/0731eb80ac2f4eae82a8aee6454c3840/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:11.197133 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd14f87-1bcc-4141-a8df-3201acf28ac1 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:11.212695 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-95e2f1a6-5c4f-4b90-9ea6-d940d2bd5f5e None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:11.214654 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-95e2f1a6-5c4f-4b90-9ea6-d940d2bd5f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:11.277471 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:11.282290 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd14f87-1bcc-4141-a8df-3201acf28ac1 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:11.298385 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 810/1605] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:11.298385 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2f1a6-5c4f-4b90-9ea6-d940d2bd5f5e None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:11.312918 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2f1a6-5c4f-4b90-9ea6-d940d2bd5f5e None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:11.319845 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf552879-d080-496b-be40-dcbd95fe4e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:11.322427 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf552879-d080-496b-be40-dcbd95fe4e8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:11.323310 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf552879-d080-496b-be40-dcbd95fe4e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:11.324040 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf552879-d080-496b-be40-dcbd95fe4e8c admin admin] PATH_INFO: `/v3/projects/9351d2e0e0e14585820951a9398b35f4/users/19457c759f0b4f5cbc530364af1f47d7/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:11.347550 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf552879-d080-496b-be40-dcbd95fe4e8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9351d2e0e0e14585820951a9398b35f4, user_id=19457c759f0b4f5cbc530364af1f47d7, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:11.350530 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf552879-d080-496b-be40-dcbd95fe4e8c admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:11.359832 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff11c2-fef7-4933-b8f8-0169f5e963bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:11.361363 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff11c2-fef7-4933-b8f8-0169f5e963bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:11.361632 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff11c2-fef7-4933-b8f8-0169f5e963bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:11.361906 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff11c2-fef7-4933-b8f8-0169f5e963bb admin admin] PATH_INFO: `/v3/projects/082b5523ee504be09f6fcaf833207a4f/users/a094273e734342eca2bdf159fab8a8d2/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:11.365291 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:11.369185 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 811/1606] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:23:11 2022] PUT /identity/v3/projects/9351d2e0e0e14585820951a9398b35f4/users/19457c759f0b4f5cbc530364af1f47d7/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:11.375429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf552879-d080-496b-be40-dcbd95fe4e8c admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:11.382252 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aff11c2-fef7-4933-b8f8-0169f5e963bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=082b5523ee504be09f6fcaf833207a4f, user_id=a094273e734342eca2bdf159fab8a8d2, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:11.385318 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aff11c2-fef7-4933-b8f8-0169f5e963bb admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:11.409421 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 796/1607] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:23:11 2022] PUT /identity/v3/projects/082b5523ee504be09f6fcaf833207a4f/users/a094273e734342eca2bdf159fab8a8d2/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:11.414485 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e41ec-d31a-409d-aedb-56cb289d06da None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:11.414774 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e41ec-d31a-409d-aedb-56cb289d06da None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:11.415048 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e41ec-d31a-409d-aedb-56cb289d06da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:11.436902 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-757e41ec-d31a-409d-aedb-56cb289d06da None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:11.438158 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-757e41ec-d31a-409d-aedb-56cb289d06da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:11.461657 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf552879-d080-496b-be40-dcbd95fe4e8c admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:11.512124 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 797/1608] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:11.512124 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e41ec-d31a-409d-aedb-56cb289d06da None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:11.520043 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e41ec-d31a-409d-aedb-56cb289d06da None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:11.522992 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61bb577-c899-469f-864c-79d46e2a4dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:11.524137 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61bb577-c899-469f-864c-79d46e2a4dee admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:11.524531 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61bb577-c899-469f-864c-79d46e2a4dee admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:11.524901 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61bb577-c899-469f-864c-79d46e2a4dee admin admin] PATH_INFO: `/v3/projects/76d3d40037e44f2091868229c4318db4/users/8054d4c2d4bc4969a93a133158db767e/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:11.525753 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4eceb8-481d-407c-89db-d9926481b0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:11.526550 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4eceb8-481d-407c-89db-d9926481b0d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:11.526834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4eceb8-481d-407c-89db-d9926481b0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:11.527116 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4eceb8-481d-407c-89db-d9926481b0d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:11.527827 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4eceb8-481d-407c-89db-d9926481b0d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:11.529817 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4eceb8-481d-407c-89db-d9926481b0d5 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:11.542624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61bb577-c899-469f-864c-79d46e2a4dee admin admin] RBAC: Authorizing `identity:create_grant(project_id=76d3d40037e44f2091868229c4318db4, user_id=8054d4c2d4bc4969a93a133158db767e, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:11.545560 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61bb577-c899-469f-864c-79d46e2a4dee admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:11.553854 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 798/1609] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:23:11 2022] POST /identity/v3/users => generated 292 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:11.555666 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f71d3-fb55-4577-a6c1-d1bc5dcd6e08 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:11.555953 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f71d3-fb55-4577-a6c1-d1bc5dcd6e08 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:11.556243 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f71d3-fb55-4577-a6c1-d1bc5dcd6e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:11.573477 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 812/1610] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:23:11 2022] PUT /identity/v3/projects/76d3d40037e44f2091868229c4318db4/users/8054d4c2d4bc4969a93a133158db767e/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:11.573477 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61bb577-c899-469f-864c-79d46e2a4dee admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:11.588503 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-839f71d3-fb55-4577-a6c1-d1bc5dcd6e08 None None] MFA Rules not processed for user `0731eb80ac2f4eae82a8aee6454c3840`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:11.589798 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-839f71d3-fb55-4577-a6c1-d1bc5dcd6e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:11.624261 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61bb577-c899-469f-864c-79d46e2a4dee admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:11.628025 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9dfefd-a99d-4a14-9411-9cac9dcad928 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:11.628687 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9dfefd-a99d-4a14-9411-9cac9dcad928 service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:11.629706 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9dfefd-a99d-4a14-9411-9cac9dcad928 service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:11.629706 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9dfefd-a99d-4a14-9411-9cac9dcad928 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:11.637725 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:11.661360 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 799/1611] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:11.662748 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520c530-ece1-473a-aee0-9dc13d576d28 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:11.663046 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520c530-ece1-473a-aee0-9dc13d576d28 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:11.663319 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520c530-ece1-473a-aee0-9dc13d576d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:11.683596 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-5520c530-ece1-473a-aee0-9dc13d576d28 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:11.684707 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-5520c530-ece1-473a-aee0-9dc13d576d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:11.701919 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9dfefd-a99d-4a14-9411-9cac9dcad928 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:11.703777 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9dfefd-a99d-4a14-9411-9cac9dcad928 service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:11.714396 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 813/1612] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:11.715699 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f1a6c-b52a-4d2e-8248-60f09493b059 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:11.715998 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f1a6c-b52a-4d2e-8248-60f09493b059 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:11.716251 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f1a6c-b52a-4d2e-8248-60f09493b059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:11.734960 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-d87f1a6c-b52a-4d2e-8248-60f09493b059 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:11.736216 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-d87f1a6c-b52a-4d2e-8248-60f09493b059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:11.739431 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:11.748213 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 814/1613] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:11.748213 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95600b5f-a16e-4e84-989d-1634719a976c None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:11.748213 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95600b5f-a16e-4e84-989d-1634719a976c None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:11.748213 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95600b5f-a16e-4e84-989d-1634719a976c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:11.755037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 800/1614] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:11.756885 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5520c530-ece1-473a-aee0-9dc13d576d28 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:11.758271 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-5520c530-ece1-473a-aee0-9dc13d576d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:11.770944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5520c530-ece1-473a-aee0-9dc13d576d28 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:11.774759 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed394f-483d-46a6-8058-03a5436d9114 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:11.775565 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed394f-483d-46a6-8058-03a5436d9114 service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:11.775948 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed394f-483d-46a6-8058-03a5436d9114 service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:11.776321 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ed394f-483d-46a6-8058-03a5436d9114 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:11.778678 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-95600b5f-a16e-4e84-989d-1634719a976c None None] MFA Rules not processed for user `1b22fa493ae04d5fac13b94cff154eee`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:11.779874 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-95600b5f-a16e-4e84-989d-1634719a976c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:11.787322 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed394f-483d-46a6-8058-03a5436d9114 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:11.788797 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ed394f-483d-46a6-8058-03a5436d9114 service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:11.801215 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 801/1615] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:11.802247 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed394f-483d-46a6-8058-03a5436d9114 service neutron] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:11.813693 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ed394f-483d-46a6-8058-03a5436d9114 service neutron] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:11.825628 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53bde14-b846-4075-87d6-d234057825c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:11.827583 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bde14-b846-4075-87d6-d234057825c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:11.828211 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bde14-b846-4075-87d6-d234057825c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:11.828535 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53bde14-b846-4075-87d6-d234057825c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:11.830188 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53bde14-b846-4075-87d6-d234057825c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:11.833185 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53bde14-b846-4075-87d6-d234057825c2 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:11.852840 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 802/1616] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:11 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:11.852840 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef591a5c-a551-4db8-acf9-807e350a65d1 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:11.852840 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef591a5c-a551-4db8-acf9-807e350a65d1 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:11.852840 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef591a5c-a551-4db8-acf9-807e350a65d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:11.882975 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 815/1617] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:11.882975 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34915e44-1f25-4f7a-8fa4-4f3fc6a445b4 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:11.882975 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34915e44-1f25-4f7a-8fa4-4f3fc6a445b4 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:11.882975 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34915e44-1f25-4f7a-8fa4-4f3fc6a445b4 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:11.882975 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 816/1618] 10.222.0.50 () {54 vars in 961 bytes} [Fri May 27 03:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:23:11.882975 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34915e44-1f25-4f7a-8fa4-4f3fc6a445b4 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:11.893594 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-ef591a5c-a551-4db8-acf9-807e350a65d1 None None] MFA Rules not processed for user `8054d4c2d4bc4969a93a133158db767e`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:11.895366 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-ef591a5c-a551-4db8-acf9-807e350a65d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:11.925388 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34915e44-1f25-4f7a-8fa4-4f3fc6a445b4 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:11.934824 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e867c474-9ed8-4c8b-81b9-1d11ed8ec33a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:11.935492 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e867c474-9ed8-4c8b-81b9-1d11ed8ec33a service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:11.935844 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e867c474-9ed8-4c8b-81b9-1d11ed8ec33a service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:11.936106 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e867c474-9ed8-4c8b-81b9-1d11ed8ec33a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:11.952908 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e867c474-9ed8-4c8b-81b9-1d11ed8ec33a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:11.955110 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e867c474-9ed8-4c8b-81b9-1d11ed8ec33a service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:11.970050 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 817/1619] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:11.971216 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e867c474-9ed8-4c8b-81b9-1d11ed8ec33a service neutron] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:11.982176 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e867c474-9ed8-4c8b-81b9-1d11ed8ec33a service neutron] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:11.988013 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 803/1620] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:11 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:11.988621 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7696e22b-48ca-4807-ba39-a52cfe677b93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:11.989049 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7696e22b-48ca-4807-ba39-a52cfe677b93 service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:11.989266 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7696e22b-48ca-4807-ba39-a52cfe677b93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:11.989601 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7696e22b-48ca-4807-ba39-a52cfe677b93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:11.990524 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef591a5c-a551-4db8-acf9-807e350a65d1 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:12.001589 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7696e22b-48ca-4807-ba39-a52cfe677b93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:12.003263 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7696e22b-48ca-4807-ba39-a52cfe677b93 service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:12.004050 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef591a5c-a551-4db8-acf9-807e350a65d1 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:12.012612 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a92d7f-8888-4f62-977d-461608fdfc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:12.013858 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a92d7f-8888-4f62-977d-461608fdfc10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:12.014197 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a92d7f-8888-4f62-977d-461608fdfc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:12.014479 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a92d7f-8888-4f62-977d-461608fdfc10 admin admin] PATH_INFO: `/v3/projects/0a1ce1a56b214a5eba331c293ecccb11/users/a30360dc68f548e5a902694d5d895375/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:12.017006 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 818/1621] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:12.018909 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d97b15-5395-40df-a472-aa3906cbdfe6 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:12.019509 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d97b15-5395-40df-a472-aa3906cbdfe6 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:12.019509 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d97b15-5395-40df-a472-aa3906cbdfe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:12.034343 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a92d7f-8888-4f62-977d-461608fdfc10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a1ce1a56b214a5eba331c293ecccb11, user_id=a30360dc68f548e5a902694d5d895375, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:12.037729 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a92d7f-8888-4f62-977d-461608fdfc10 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:12.039670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-29d97b15-5395-40df-a472-aa3906cbdfe6 None None] MFA Rules not processed for user `1b22fa493ae04d5fac13b94cff154eee`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:12.041626 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-29d97b15-5395-40df-a472-aa3906cbdfe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:12.051855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 819/1622] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:12.053345 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c69fda-9067-4a97-aec2-ae58b3373c4f None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:12.053723 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c69fda-9067-4a97-aec2-ae58b3373c4f None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:12.057016 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c69fda-9067-4a97-aec2-ae58b3373c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:12.060618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 804/1623] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:23:11 2022] PUT /identity/v3/projects/0a1ce1a56b214a5eba331c293ecccb11/users/a30360dc68f548e5a902694d5d895375/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:12.061883 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a92d7f-8888-4f62-977d-461608fdfc10 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:12.079794 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-66c69fda-9067-4a97-aec2-ae58b3373c4f None None] MFA Rules not processed for user `1a699988f08a4001b3b7d764f01f4fc5`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:12.081948 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-66c69fda-9067-4a97-aec2-ae58b3373c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:12.138367 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:12.154625 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a92d7f-8888-4f62-977d-461608fdfc10 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:12.160380 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e03d1-4817-43be-8ebc-9a1e3e38dc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:12.161477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e03d1-4817-43be-8ebc-9a1e3e38dc91 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:12.161919 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e03d1-4817-43be-8ebc-9a1e3e38dc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:12.162215 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e03d1-4817-43be-8ebc-9a1e3e38dc91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:12.163018 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775e03d1-4817-43be-8ebc-9a1e3e38dc91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:12.163527 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 820/1624] 10.222.0.50 () {58 vars in 1058 bytes} [Fri May 27 03:23:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:12.164465 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c69fda-9067-4a97-aec2-ae58b3373c4f None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:12.174558 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775e03d1-4817-43be-8ebc-9a1e3e38dc91 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:12.180995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c69fda-9067-4a97-aec2-ae58b3373c4f None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:12.189266 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5344b-93a2-4db9-9464-fb3e9b7aec38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:12.190076 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5344b-93a2-4db9-9464-fb3e9b7aec38 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:12.190468 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5344b-93a2-4db9-9464-fb3e9b7aec38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:12.191689 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5344b-93a2-4db9-9464-fb3e9b7aec38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:12.192526 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af5344b-93a2-4db9-9464-fb3e9b7aec38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:12.196457 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af5344b-93a2-4db9-9464-fb3e9b7aec38 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:12.216420 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 821/1625] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:12.218333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29553652-20e8-4a18-b4a0-b9ce1d31e1fb None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:12.218707 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29553652-20e8-4a18-b4a0-b9ce1d31e1fb None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:12.219071 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29553652-20e8-4a18-b4a0-b9ce1d31e1fb None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:12.220939 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 822/1626] 10.222.0.50 () {54 vars in 961 bytes} [Fri May 27 03:23:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:23:12.222348 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29553652-20e8-4a18-b4a0-b9ce1d31e1fb None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:12.225001 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 805/1627] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:23:12 2022] POST /identity/v3/projects => generated 403 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:12.225967 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e03d1-4817-43be-8ebc-9a1e3e38dc91 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:12.268252 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:12.336762 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e03d1-4817-43be-8ebc-9a1e3e38dc91 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:12.345655 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f859f8-f0cc-44b8-8990-6c63ce4b57ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:12.347300 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f859f8-f0cc-44b8-8990-6c63ce4b57ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:12.347641 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f859f8-f0cc-44b8-8990-6c63ce4b57ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:12.347977 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f859f8-f0cc-44b8-8990-6c63ce4b57ef admin admin] PATH_INFO: `/v3/projects/0a1ce1a56b214a5eba331c293ecccb11/users/a30360dc68f548e5a902694d5d895375/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:12.350733 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29553652-20e8-4a18-b4a0-b9ce1d31e1fb None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:12.350931 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f859f8-f0cc-44b8-8990-6c63ce4b57ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a1ce1a56b214a5eba331c293ecccb11, user_id=a30360dc68f548e5a902694d5d895375, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:12.355247 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f859f8-f0cc-44b8-8990-6c63ce4b57ef admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:12.357923 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b999691-af6c-459f-b9f9-24ab563f9003 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:12.358869 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b999691-af6c-459f-b9f9-24ab563f9003 service nova] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:12.359440 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b999691-af6c-459f-b9f9-24ab563f9003 service nova] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:12.360244 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b999691-af6c-459f-b9f9-24ab563f9003 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:12.382548 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 806/1628] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:23:12 2022] PUT /identity/v3/projects/0a1ce1a56b214a5eba331c293ecccb11/users/a30360dc68f548e5a902694d5d895375/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:12.384890 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f859f8-f0cc-44b8-8990-6c63ce4b57ef admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:12.442314 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b999691-af6c-459f-b9f9-24ab563f9003 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:12.444366 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b999691-af6c-459f-b9f9-24ab563f9003 service nova] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:12.467662 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 823/1629] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:12.469325 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa351f-704a-4a09-9b9a-1567fce5559f None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:12.473675 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa351f-704a-4a09-9b9a-1567fce5559f None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:12.474017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa351f-704a-4a09-9b9a-1567fce5559f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:12.493205 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f859f8-f0cc-44b8-8990-6c63ce4b57ef admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:12.499395 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84665165-5479-4e46-8aa7-374121df3ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:12.500221 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84665165-5479-4e46-8aa7-374121df3ccd admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:12.500569 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84665165-5479-4e46-8aa7-374121df3ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:12.500904 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84665165-5479-4e46-8aa7-374121df3ccd admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:12.502118 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84665165-5479-4e46-8aa7-374121df3ccd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:12.504300 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84665165-5479-4e46-8aa7-374121df3ccd admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:12.508874 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-baaa351f-704a-4a09-9b9a-1567fce5559f None None] MFA Rules not processed for user `1a699988f08a4001b3b7d764f01f4fc5`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:12.511325 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-baaa351f-704a-4a09-9b9a-1567fce5559f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:12.542510 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 807/1630] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:23:12 2022] POST /identity/v3/users => generated 308 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:12.547678 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070b358-7980-4a0c-997b-b3799a54a521 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:12.548061 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070b358-7980-4a0c-997b-b3799a54a521 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:12.549249 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070b358-7980-4a0c-997b-b3799a54a521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:12.605963 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 824/1631] 10.222.0.50 () {58 vars in 1058 bytes} [Fri May 27 03:23:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:12.606936 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa351f-704a-4a09-9b9a-1567fce5559f None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:12.613309 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:12.618164 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa351f-704a-4a09-9b9a-1567fce5559f None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:12.618353 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-e070b358-7980-4a0c-997b-b3799a54a521 None None] MFA Rules not processed for user `a30360dc68f548e5a902694d5d895375`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:12.619705 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-e070b358-7980-4a0c-997b-b3799a54a521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:12.628565 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb249e9f-c75b-4a5d-b6e3-ccd46914e8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:12.629284 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb249e9f-c75b-4a5d-b6e3-ccd46914e8ef admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:12.629561 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb249e9f-c75b-4a5d-b6e3-ccd46914e8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:12.629828 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb249e9f-c75b-4a5d-b6e3-ccd46914e8ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:12.630547 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb249e9f-c75b-4a5d-b6e3-ccd46914e8ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:12.633256 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb249e9f-c75b-4a5d-b6e3-ccd46914e8ef admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:12.657553 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 825/1632] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:12.659671 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb249e9f-c75b-4a5d-b6e3-ccd46914e8ef admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:12.690131 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb249e9f-c75b-4a5d-b6e3-ccd46914e8ef admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:12.708151 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e087e2-1415-4dfa-9522-e0d3a6ef2901 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:12.711127 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e087e2-1415-4dfa-9522-e0d3a6ef2901 service nova] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:12.711542 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e087e2-1415-4dfa-9522-e0d3a6ef2901 service nova] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:12.711828 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e087e2-1415-4dfa-9522-e0d3a6ef2901 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:12.747826 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 808/1633] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:12 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:12.749731 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e070b358-7980-4a0c-997b-b3799a54a521 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:12.777805 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e070b358-7980-4a0c-997b-b3799a54a521 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:12.788843 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05953650-7efa-404d-8e95-ffc11a31caab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:12.791183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05953650-7efa-404d-8e95-ffc11a31caab admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:12.791312 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05953650-7efa-404d-8e95-ffc11a31caab admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:12.792082 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05953650-7efa-404d-8e95-ffc11a31caab admin admin] PATH_INFO: `/v3/projects/a99e18940a8943adaad20cdeadae1d1b/users/65f2b47bb2d54c248669899976178d13/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:12.827755 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e087e2-1415-4dfa-9522-e0d3a6ef2901 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:12.830129 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e087e2-1415-4dfa-9522-e0d3a6ef2901 service nova] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:12.830960 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05953650-7efa-404d-8e95-ffc11a31caab admin admin] RBAC: Authorizing `identity:create_grant(project_id=a99e18940a8943adaad20cdeadae1d1b, user_id=65f2b47bb2d54c248669899976178d13, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:12.835099 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05953650-7efa-404d-8e95-ffc11a31caab admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:12.844515 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 826/1634] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:12 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:12.846800 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e087e2-1415-4dfa-9522-e0d3a6ef2901 service nova] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:12.874757 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 809/1635] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:23:12 2022] PUT /identity/v3/projects/a99e18940a8943adaad20cdeadae1d1b/users/65f2b47bb2d54c248669899976178d13/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:12.879225 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76730c08-8a58-46aa-aec8-b10f2720b323 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:12.880854 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76730c08-8a58-46aa-aec8-b10f2720b323 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:12.881246 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76730c08-8a58-46aa-aec8-b10f2720b323 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:12.884024 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 810/1636] 10.222.0.50 () {54 vars in 912 bytes} [Fri May 27 03:23:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:23:12.887796 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76730c08-8a58-46aa-aec8-b10f2720b323 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:12.916997 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:12.979094 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e087e2-1415-4dfa-9522-e0d3a6ef2901 service nova] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:13.045074 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76730c08-8a58-46aa-aec8-b10f2720b323 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:13.054340 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4f5030-ac7d-49a1-98fd-5ee49955a550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:13.055145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f5030-ac7d-49a1-98fd-5ee49955a550 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:13.055339 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f5030-ac7d-49a1-98fd-5ee49955a550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:13.055612 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f5030-ac7d-49a1-98fd-5ee49955a550 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:13.056345 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4f5030-ac7d-49a1-98fd-5ee49955a550 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:13.058075 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4f5030-ac7d-49a1-98fd-5ee49955a550 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:13.085064 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 811/1637] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:13.087059 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d171fe5-0883-4ec7-9038-d7dda2c17b3c None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:13.087366 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d171fe5-0883-4ec7-9038-d7dda2c17b3c None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:13.087790 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d171fe5-0883-4ec7-9038-d7dda2c17b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:13.092850 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a616d-aa2f-4a07-90c4-0366a275c9a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:13.093553 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a616d-aa2f-4a07-90c4-0366a275c9a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:13.093833 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a616d-aa2f-4a07-90c4-0366a275c9a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:13.094104 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a616d-aa2f-4a07-90c4-0366a275c9a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:13.133561 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-1d171fe5-0883-4ec7-9038-d7dda2c17b3c None None] MFA Rules not processed for user `1a699988f08a4001b3b7d764f01f4fc5`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:13.135014 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-1d171fe5-0883-4ec7-9038-d7dda2c17b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:13.239938 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4a616d-aa2f-4a07-90c4-0366a275c9a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:13.241477 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4a616d-aa2f-4a07-90c4-0366a275c9a0 service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:13.251683 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:13.263705 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 827/1638] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:12 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:13.264528 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a616d-aa2f-4a07-90c4-0366a275c9a0 service cinder] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:13.289318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a616d-aa2f-4a07-90c4-0366a275c9a0 service cinder] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:13.303935 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 812/1639] 10.222.0.50 () {58 vars in 1009 bytes} [Fri May 27 03:23:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:13.306380 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0eef6a-f91d-45be-af7f-b03f9fa1e779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:13.307638 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0eef6a-f91d-45be-af7f-b03f9fa1e779 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:13.308888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0eef6a-f91d-45be-af7f-b03f9fa1e779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:13.308888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0eef6a-f91d-45be-af7f-b03f9fa1e779 admin admin] PATH_INFO: `/v3/projects/a99e18940a8943adaad20cdeadae1d1b/users/65f2b47bb2d54c248669899976178d13/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:13.310599 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0eef6a-f91d-45be-af7f-b03f9fa1e779 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a99e18940a8943adaad20cdeadae1d1b, user_id=65f2b47bb2d54c248669899976178d13, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:13.314297 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0eef6a-f91d-45be-af7f-b03f9fa1e779 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:13.331349 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 828/1640] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:23:13 2022] PUT /identity/v3/projects/a99e18940a8943adaad20cdeadae1d1b/users/65f2b47bb2d54c248669899976178d13/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:13.340665 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f196d41-1b3b-4b8e-80f3-3cdf35dda09a None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:13.340942 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f196d41-1b3b-4b8e-80f3-3cdf35dda09a None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:13.341843 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f196d41-1b3b-4b8e-80f3-3cdf35dda09a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:13.351152 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0eef6a-f91d-45be-af7f-b03f9fa1e779 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:13.449380 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-2f196d41-1b3b-4b8e-80f3-3cdf35dda09a None None] MFA Rules not processed for user `65f2b47bb2d54c248669899976178d13`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:13.457717 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-2f196d41-1b3b-4b8e-80f3-3cdf35dda09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:13.564473 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0eef6a-f91d-45be-af7f-b03f9fa1e779 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:13.569669 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef581204-71cc-4117-a44e-1a98aa79f5fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:13.571705 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef581204-71cc-4117-a44e-1a98aa79f5fc service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:13.572219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef581204-71cc-4117-a44e-1a98aa79f5fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:13.573833 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef581204-71cc-4117-a44e-1a98aa79f5fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:13.647935 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 813/1641] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:13 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:23:13.651815 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddabb1e-baae-4612-afe6-88661b0c27a0 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:13.651815 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddabb1e-baae-4612-afe6-88661b0c27a0 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:13.652120 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddabb1e-baae-4612-afe6-88661b0c27a0 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:13.662595 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 814/1642] 10.222.0.50 () {54 vars in 912 bytes} [Fri May 27 03:23:13 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:23:13.664563 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aebab24-3e38-4656-897b-10996e987d27 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:13.664903 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aebab24-3e38-4656-897b-10996e987d27 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:13.666660 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aebab24-3e38-4656-897b-10996e987d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:13.699345 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef581204-71cc-4117-a44e-1a98aa79f5fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:13.699614 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-4aebab24-3e38-4656-897b-10996e987d27 None None] MFA Rules not processed for user `1a699988f08a4001b3b7d764f01f4fc5`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:13.701392 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef581204-71cc-4117-a44e-1a98aa79f5fc service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:13.702736 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-4aebab24-3e38-4656-897b-10996e987d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:13.728739 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 815/1643] 10.222.0.50 () {58 vars in 1009 bytes} [Fri May 27 03:23:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:13.729675 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aebab24-3e38-4656-897b-10996e987d27 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:13.735784 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 829/1644] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:13.743081 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef581204-71cc-4117-a44e-1a98aa79f5fc service cinder] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:13.745346 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-ef581204-71cc-4117-a44e-1a98aa79f5fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:13.749495 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aebab24-3e38-4656-897b-10996e987d27 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:13.757606 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fbf124-8ca9-41d9-8e14-eeac36bfea0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:13.758798 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fbf124-8ca9-41d9-8e14-eeac36bfea0e service cinder] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:13.758798 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fbf124-8ca9-41d9-8e14-eeac36bfea0e service cinder] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:13.759065 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fbf124-8ca9-41d9-8e14-eeac36bfea0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:13.775552 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fbf124-8ca9-41d9-8e14-eeac36bfea0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:13.777111 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fbf124-8ca9-41d9-8e14-eeac36bfea0e service cinder] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:13.807479 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 816/1645] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:13 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:13.845162 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef581204-71cc-4117-a44e-1a98aa79f5fc service cinder] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:13.849426 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969524a3-b211-44e5-8f0a-ba182232d69c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3c9c5c8d2e4376b1b70d974d4cb851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:13.850106 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969524a3-b211-44e5-8f0a-ba182232d69c service glance] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:13.850438 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969524a3-b211-44e5-8f0a-ba182232d69c service glance] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:13.850713 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969524a3-b211-44e5-8f0a-ba182232d69c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:13.863860 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969524a3-b211-44e5-8f0a-ba182232d69c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:13.865477 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969524a3-b211-44e5-8f0a-ba182232d69c service glance] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:13.881827 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 830/1646] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:14.517646 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fbf124-8ca9-41d9-8e14-eeac36bfea0e service cinder] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:14.523163 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:14.646799 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fbf124-8ca9-41d9-8e14-eeac36bfea0e service cinder] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:14.653116 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbb36b5-4aca-45ae-8ff1-edbd284730c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:14.654199 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb36b5-4aca-45ae-8ff1-edbd284730c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:14.654480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb36b5-4aca-45ae-8ff1-edbd284730c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:14.654752 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbb36b5-4aca-45ae-8ff1-edbd284730c7 admin admin] PATH_INFO: `/v3/users/8054d4c2d4bc4969a93a133158db767e` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:14.656072 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbb36b5-4aca-45ae-8ff1-edbd284730c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8054d4c2d4bc4969a93a133158db767e)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:14.657138 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: WARNING py.warnings [None req-bbbb36b5-4aca-45ae-8ff1-edbd284730c7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:23:14.657138 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: warnings.warn(msg) May 27 03:23:14.657138 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: May 27 03:23:14.668130 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbb36b5-4aca-45ae-8ff1-edbd284730c7 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:14.811696 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-bbbb36b5-4aca-45ae-8ff1-edbd284730c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8054d4c2d4bc4969a93a133158db767e'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:14.835217 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969524a3-b211-44e5-8f0a-ba182232d69c service glance] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:14.855238 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969524a3-b211-44e5-8f0a-ba182232d69c service glance] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:14.861091 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-bbbb36b5-4aca-45ae-8ff1-edbd284730c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8054d4c2d4bc4969a93a133158db767e'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:14.882512 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7083c5f1-3be7-4dcb-b8d1-fd32cee02af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:14.884919 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-bbbb36b5-4aca-45ae-8ff1-edbd284730c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8054d4c2d4bc4969a93a133158db767e'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:14.887822 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083c5f1-3be7-4dcb-b8d1-fd32cee02af9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:14.888133 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083c5f1-3be7-4dcb-b8d1-fd32cee02af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:14.888415 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7083c5f1-3be7-4dcb-b8d1-fd32cee02af9 admin admin] PATH_INFO: `/v3/users/8054e514b45e42f38a6713ba1d823cec` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:14.895117 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7083c5f1-3be7-4dcb-b8d1-fd32cee02af9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8054e514b45e42f38a6713ba1d823cec)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:14.897185 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7083c5f1-3be7-4dcb-b8d1-fd32cee02af9 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:14.897944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-bbbb36b5-4aca-45ae-8ff1-edbd284730c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8054d4c2d4bc4969a93a133158db767e'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:14.898764 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-bbbb36b5-4aca-45ae-8ff1-edbd284730c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8054d4c2d4bc4969a93a133158db767e'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:14.906293 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 817/1647] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:23:14 2022] DELETE /identity/v3/users/8054d4c2d4bc4969a93a133158db767e => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:14.910555 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08835c58-6d93-4dfe-abf2-541fa00b0bc5 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:14.910919 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08835c58-6d93-4dfe-abf2-541fa00b0bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:14.911253 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08835c58-6d93-4dfe-abf2-541fa00b0bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:14.934043 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-08835c58-6d93-4dfe-abf2-541fa00b0bc5 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:14.935475 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-08835c58-6d93-4dfe-abf2-541fa00b0bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:14.967916 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:15.025694 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 818/1648] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:23:15.025694 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9addb429-10a7-4889-a45c-415fbe310b08 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:15.025694 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9addb429-10a7-4889-a45c-415fbe310b08 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:15.025694 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9addb429-10a7-4889-a45c-415fbe310b08 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:15.026616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 819/1649] 10.222.0.50 () {54 vars in 912 bytes} [Fri May 27 03:23:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:23:15.027307 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-7083c5f1-3be7-4dcb-b8d1-fd32cee02af9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8054e514b45e42f38a6713ba1d823cec'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:15.030983 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-7083c5f1-3be7-4dcb-b8d1-fd32cee02af9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8054e514b45e42f38a6713ba1d823cec'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:15.033758 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9172c62-e484-4e73-aa24-af8a83b7446e None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:15.034162 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9172c62-e484-4e73-aa24-af8a83b7446e None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:15.034602 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9172c62-e484-4e73-aa24-af8a83b7446e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:15.045117 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-7083c5f1-3be7-4dcb-b8d1-fd32cee02af9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8054e514b45e42f38a6713ba1d823cec'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:15.045882 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-7083c5f1-3be7-4dcb-b8d1-fd32cee02af9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8054e514b45e42f38a6713ba1d823cec'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:15.073271 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-7083c5f1-3be7-4dcb-b8d1-fd32cee02af9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8054e514b45e42f38a6713ba1d823cec'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:15.107966 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 831/1650] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:23:14 2022] DELETE /identity/v3/users/8054e514b45e42f38a6713ba1d823cec => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:15.109845 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f63e7a-e594-4c6e-9c50-4b3fa80043e4 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:15.110139 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f63e7a-e594-4c6e-9c50-4b3fa80043e4 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:15.110547 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f63e7a-e594-4c6e-9c50-4b3fa80043e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:15.117687 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-d9172c62-e484-4e73-aa24-af8a83b7446e None None] MFA Rules not processed for user `4ee30c0d6d694ee8945800157a9fc5b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:15.120309 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-d9172c62-e484-4e73-aa24-af8a83b7446e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:15.134008 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:15.149831 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-83f63e7a-e594-4c6e-9c50-4b3fa80043e4 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:15.151581 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-83f63e7a-e594-4c6e-9c50-4b3fa80043e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:15.236646 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 832/1651] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:23:15.237895 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f63e7a-e594-4c6e-9c50-4b3fa80043e4 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:15.239342 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-83f63e7a-e594-4c6e-9c50-4b3fa80043e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:15.364012 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f63e7a-e594-4c6e-9c50-4b3fa80043e4 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:15.368332 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45a119-6b90-4b97-949c-5b88c87b9c32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:15.369068 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45a119-6b90-4b97-949c-5b88c87b9c32 service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:15.369407 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45a119-6b90-4b97-949c-5b88c87b9c32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:15.369632 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45a119-6b90-4b97-949c-5b88c87b9c32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:15.371703 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-2f45a119-6b90-4b97-949c-5b88c87b9c32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:15.395320 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 820/1652] 10.222.0.50 () {58 vars in 1009 bytes} [Fri May 27 03:23:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:15.396551 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9172c62-e484-4e73-aa24-af8a83b7446e None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:15.397648 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f45a119-6b90-4b97-949c-5b88c87b9c32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:15.399127 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f45a119-6b90-4b97-949c-5b88c87b9c32 service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:15.406091 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9172c62-e484-4e73-aa24-af8a83b7446e None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:15.410199 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4dc529-418a-4745-9419-56f8d716d737 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:15.411209 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4dc529-418a-4745-9419-56f8d716d737 service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:15.412358 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4dc529-418a-4745-9419-56f8d716d737 service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:15.412761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4dc529-418a-4745-9419-56f8d716d737 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:15.420436 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 833/1653] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:15.426224 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4dc529-418a-4745-9419-56f8d716d737 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:15.427970 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4dc529-418a-4745-9419-56f8d716d737 service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:15.440662 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 821/1654] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:15.458675 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45a119-6b90-4b97-949c-5b88c87b9c32 service neutron] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:15.471204 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45a119-6b90-4b97-949c-5b88c87b9c32 service neutron] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:15.477923 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57c16a0-158c-44c5-bdca-60309c10da73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:15.478985 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c16a0-158c-44c5-bdca-60309c10da73 service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:15.479369 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c16a0-158c-44c5-bdca-60309c10da73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:15.479757 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57c16a0-158c-44c5-bdca-60309c10da73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:15.488713 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57c16a0-158c-44c5-bdca-60309c10da73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:15.492343 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57c16a0-158c-44c5-bdca-60309c10da73 service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:15.504640 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 834/1655] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:16.473261 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4dc529-418a-4745-9419-56f8d716d737 service neutron] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:16.476314 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-1c4dc529-418a-4745-9419-56f8d716d737 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:16.527629 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4dc529-418a-4745-9419-56f8d716d737 service neutron] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:16.545280 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d94c2b-3a57-4104-a3e6-b072c9e5f310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:16.551953 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d94c2b-3a57-4104-a3e6-b072c9e5f310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:16.552327 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d94c2b-3a57-4104-a3e6-b072c9e5f310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:16.552523 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d94c2b-3a57-4104-a3e6-b072c9e5f310 admin admin] PATH_INFO: `/v3/projects/b5bae16346a94be89a5af009f5937fd6` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:16.553910 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d94c2b-3a57-4104-a3e6-b072c9e5f310 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5bae16346a94be89a5af009f5937fd6)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:16.560713 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d94c2b-3a57-4104-a3e6-b072c9e5f310 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:16.689418 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-b6d94c2b-3a57-4104-a3e6-b072c9e5f310 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5bae16346a94be89a5af009f5937fd6', 'request_id': 'req-b6d94c2b-3a57-4104-a3e6-b072c9e5f310'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:16.703669 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-b6d94c2b-3a57-4104-a3e6-b072c9e5f310 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5bae16346a94be89a5af009f5937fd6', 'request_id': 'req-b6d94c2b-3a57-4104-a3e6-b072c9e5f310'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:16.717628 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-b6d94c2b-3a57-4104-a3e6-b072c9e5f310 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5bae16346a94be89a5af009f5937fd6', 'request_id': 'req-b6d94c2b-3a57-4104-a3e6-b072c9e5f310'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:16.765097 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-b6d94c2b-3a57-4104-a3e6-b072c9e5f310 admin admin] The token cache is being invalidate because project b5bae16346a94be89a5af009f5937fd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:23:16.765429 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-b6d94c2b-3a57-4104-a3e6-b072c9e5f310 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:16.768891 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-b6d94c2b-3a57-4104-a3e6-b072c9e5f310 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:16.771337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 822/1656] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:23:16 2022] DELETE /identity/v3/projects/b5bae16346a94be89a5af009f5937fd6 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:16.776393 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57c16a0-158c-44c5-bdca-60309c10da73 service neutron] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:16.794123 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57c16a0-158c-44c5-bdca-60309c10da73 service neutron] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:16.924433 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:16.998504 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d73f1fb-209c-464a-aa0d-e6033f7ce1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:17.002690 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d73f1fb-209c-464a-aa0d-e6033f7ce1b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:17.003077 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d73f1fb-209c-464a-aa0d-e6033f7ce1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:17.003539 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d73f1fb-209c-464a-aa0d-e6033f7ce1b2 admin admin] PATH_INFO: `/v3/users/a30360dc68f548e5a902694d5d895375` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:17.005033 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d73f1fb-209c-464a-aa0d-e6033f7ce1b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a30360dc68f548e5a902694d5d895375)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:17.014823 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d73f1fb-209c-464a-aa0d-e6033f7ce1b2 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:17.146471 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-9d73f1fb-209c-464a-aa0d-e6033f7ce1b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a30360dc68f548e5a902694d5d895375'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:17.149363 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-9d73f1fb-209c-464a-aa0d-e6033f7ce1b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a30360dc68f548e5a902694d5d895375'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:17.168990 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-9d73f1fb-209c-464a-aa0d-e6033f7ce1b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a30360dc68f548e5a902694d5d895375'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:17.169847 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-9d73f1fb-209c-464a-aa0d-e6033f7ce1b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a30360dc68f548e5a902694d5d895375'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:17.194006 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-9d73f1fb-209c-464a-aa0d-e6033f7ce1b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a30360dc68f548e5a902694d5d895375'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:17.231527 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 835/1657] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:23:16 2022] DELETE /identity/v3/users/a30360dc68f548e5a902694d5d895375 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:17.233393 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71255328-067d-483c-89ca-c83b0ee9ade2 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:17.233824 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71255328-067d-483c-89ca-c83b0ee9ade2 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:17.236279 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71255328-067d-483c-89ca-c83b0ee9ade2 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:17.238055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 823/1658] 10.222.0.50 () {54 vars in 960 bytes} [Fri May 27 03:23:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:23:17.245505 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea0c9c-45c8-41cd-8806-9768b5670bab None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:17.245911 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea0c9c-45c8-41cd-8806-9768b5670bab None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:17.246341 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea0c9c-45c8-41cd-8806-9768b5670bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:17.286886 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-ebea0c9c-45c8-41cd-8806-9768b5670bab None None] MFA Rules not processed for user `30963f2f464e4f54a8aea0175370a235`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:17.288317 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-ebea0c9c-45c8-41cd-8806-9768b5670bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:17.339898 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71255328-067d-483c-89ca-c83b0ee9ade2 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:17.343096 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-71255328-067d-483c-89ca-c83b0ee9ade2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:17.484754 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 836/1659] 10.222.0.50 () {58 vars in 1057 bytes} [Fri May 27 03:23:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:17.493512 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea0c9c-45c8-41cd-8806-9768b5670bab None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:17.517843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea0c9c-45c8-41cd-8806-9768b5670bab None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:17.520279 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71255328-067d-483c-89ca-c83b0ee9ade2 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:17.531817 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bd2df1-f4c1-426b-b71c-b0e75d879fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:17.532996 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd2df1-f4c1-426b-b71c-b0e75d879fd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:17.533499 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd2df1-f4c1-426b-b71c-b0e75d879fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:17.533600 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd2df1-f4c1-426b-b71c-b0e75d879fd2 admin admin] PATH_INFO: `/v3/projects/76d3d40037e44f2091868229c4318db4` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:17.534782 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdbf312-7813-46a6-857e-256c5dd138f9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30963f2f464e4f54a8aea0175370a235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:17.535423 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdbf312-7813-46a6-857e-256c5dd138f9 service placement] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:17.536101 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdbf312-7813-46a6-857e-256c5dd138f9 service placement] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:17.536221 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bd2df1-f4c1-426b-b71c-b0e75d879fd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76d3d40037e44f2091868229c4318db4)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:17.536290 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdbf312-7813-46a6-857e-256c5dd138f9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:17.538655 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bd2df1-f4c1-426b-b71c-b0e75d879fd2 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:17.542130 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-9fdbf312-7813-46a6-857e-256c5dd138f9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:17.569610 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdbf312-7813-46a6-857e-256c5dd138f9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:17.571439 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdbf312-7813-46a6-857e-256c5dd138f9 service placement] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:17.608569 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 837/1660] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:17.644549 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:17.671863 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-f0bd2df1-f4c1-426b-b71c-b0e75d879fd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76d3d40037e44f2091868229c4318db4', 'request_id': 'req-f0bd2df1-f4c1-426b-b71c-b0e75d879fd2'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:17.679753 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-f0bd2df1-f4c1-426b-b71c-b0e75d879fd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76d3d40037e44f2091868229c4318db4', 'request_id': 'req-f0bd2df1-f4c1-426b-b71c-b0e75d879fd2'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:17.689171 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-f0bd2df1-f4c1-426b-b71c-b0e75d879fd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76d3d40037e44f2091868229c4318db4', 'request_id': 'req-f0bd2df1-f4c1-426b-b71c-b0e75d879fd2'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:17.714069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-f0bd2df1-f4c1-426b-b71c-b0e75d879fd2 admin admin] The token cache is being invalidate because project 76d3d40037e44f2091868229c4318db4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:23:17.714430 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-f0bd2df1-f4c1-426b-b71c-b0e75d879fd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:17.715487 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-f0bd2df1-f4c1-426b-b71c-b0e75d879fd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:17.721090 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 824/1661] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:23:17 2022] DELETE /identity/v3/projects/76d3d40037e44f2091868229c4318db4 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:17.722373 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdbf312-7813-46a6-857e-256c5dd138f9 service placement] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:17.742806 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdbf312-7813-46a6-857e-256c5dd138f9 service placement] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:17.809615 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:17.886997 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8bfdfb-3fae-47ef-99dc-de56f568f84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:17.888237 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8bfdfb-3fae-47ef-99dc-de56f568f84a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:17.888353 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8bfdfb-3fae-47ef-99dc-de56f568f84a admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:17.897053 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8bfdfb-3fae-47ef-99dc-de56f568f84a admin admin] PATH_INFO: `/v3/users/65f2b47bb2d54c248669899976178d13` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:17.913351 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8bfdfb-3fae-47ef-99dc-de56f568f84a admin admin] RBAC: Authorizing `identity:delete_user(user_id=65f2b47bb2d54c248669899976178d13)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:17.915629 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8bfdfb-3fae-47ef-99dc-de56f568f84a admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:18.099943 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-7e8bfdfb-3fae-47ef-99dc-de56f568f84a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65f2b47bb2d54c248669899976178d13'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:18.101923 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-7e8bfdfb-3fae-47ef-99dc-de56f568f84a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65f2b47bb2d54c248669899976178d13'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:18.118529 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-7e8bfdfb-3fae-47ef-99dc-de56f568f84a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65f2b47bb2d54c248669899976178d13'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:18.122028 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-7e8bfdfb-3fae-47ef-99dc-de56f568f84a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65f2b47bb2d54c248669899976178d13'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:18.141003 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-7e8bfdfb-3fae-47ef-99dc-de56f568f84a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65f2b47bb2d54c248669899976178d13'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:18.157705 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 838/1662] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:23:17 2022] DELETE /identity/v3/users/65f2b47bb2d54c248669899976178d13 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:19.217134 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bd2df1-f4c1-426b-b71c-b0e75d879fd2 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:19.218600 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-f0bd2df1-f4c1-426b-b71c-b0e75d879fd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:19.250904 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bd2df1-f4c1-426b-b71c-b0e75d879fd2 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:19.434789 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129cf7c-2ec2-4908-b4b8-86ebb6d7a68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:19.435706 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129cf7c-2ec2-4908-b4b8-86ebb6d7a68d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:19.435989 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129cf7c-2ec2-4908-b4b8-86ebb6d7a68d admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:19.436263 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129cf7c-2ec2-4908-b4b8-86ebb6d7a68d admin admin] PATH_INFO: `/v3/projects/0a1ce1a56b214a5eba331c293ecccb11` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:19.439266 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129cf7c-2ec2-4908-b4b8-86ebb6d7a68d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a1ce1a56b214a5eba331c293ecccb11)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:19.446683 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129cf7c-2ec2-4908-b4b8-86ebb6d7a68d admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:19.593810 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-1129cf7c-2ec2-4908-b4b8-86ebb6d7a68d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a1ce1a56b214a5eba331c293ecccb11', 'request_id': 'req-1129cf7c-2ec2-4908-b4b8-86ebb6d7a68d'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:19.619235 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-1129cf7c-2ec2-4908-b4b8-86ebb6d7a68d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a1ce1a56b214a5eba331c293ecccb11', 'request_id': 'req-1129cf7c-2ec2-4908-b4b8-86ebb6d7a68d'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:19.631465 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-1129cf7c-2ec2-4908-b4b8-86ebb6d7a68d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a1ce1a56b214a5eba331c293ecccb11', 'request_id': 'req-1129cf7c-2ec2-4908-b4b8-86ebb6d7a68d'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:19.668990 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-1129cf7c-2ec2-4908-b4b8-86ebb6d7a68d admin admin] The token cache is being invalidate because project 0a1ce1a56b214a5eba331c293ecccb11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:23:19.669188 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-1129cf7c-2ec2-4908-b4b8-86ebb6d7a68d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:19.672402 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-1129cf7c-2ec2-4908-b4b8-86ebb6d7a68d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:19.684003 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 825/1663] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:23:19 2022] DELETE /identity/v3/projects/0a1ce1a56b214a5eba331c293ecccb11 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:19.686356 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e57c7-b6b0-4cd4-8562-d10e5559a9b0 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:19.686904 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e57c7-b6b0-4cd4-8562-d10e5559a9b0 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:19.686989 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e57c7-b6b0-4cd4-8562-d10e5559a9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:19.751934 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-345e57c7-b6b0-4cd4-8562-d10e5559a9b0 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:19.760309 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-345e57c7-b6b0-4cd4-8562-d10e5559a9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:19.962604 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 839/1664] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:19.964503 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ba934-23d7-4416-b7ee-0d8dc055e7a1 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:19.964893 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ba934-23d7-4416-b7ee-0d8dc055e7a1 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:19.965218 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ba934-23d7-4416-b7ee-0d8dc055e7a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:20.017023 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-d75ba934-23d7-4416-b7ee-0d8dc055e7a1 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:20.018425 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-d75ba934-23d7-4416-b7ee-0d8dc055e7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:20.056653 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 826/1665] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:23:20.060996 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e57c7-b6b0-4cd4-8562-d10e5559a9b0 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:20.083050 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e57c7-b6b0-4cd4-8562-d10e5559a9b0 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:20.109198 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfbb431-81bb-4389-99d1-5dced4216f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:20.110539 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfbb431-81bb-4389-99d1-5dced4216f54 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:20.113949 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfbb431-81bb-4389-99d1-5dced4216f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:20.115916 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfbb431-81bb-4389-99d1-5dced4216f54 admin admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:20.116911 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfbb431-81bb-4389-99d1-5dced4216f54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:23:20.120622 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfbb431-81bb-4389-99d1-5dced4216f54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:20.122703 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfbb431-81bb-4389-99d1-5dced4216f54 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:20.194090 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 840/1666] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:23:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:23:20.195065 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ba934-23d7-4416-b7ee-0d8dc055e7a1 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:20.211328 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ba934-23d7-4416-b7ee-0d8dc055e7a1 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:20.230056 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993909c0-80cf-4ebf-8dd9-5a59c709e254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:20.230740 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993909c0-80cf-4ebf-8dd9-5a59c709e254 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:20.231018 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993909c0-80cf-4ebf-8dd9-5a59c709e254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:20.231294 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993909c0-80cf-4ebf-8dd9-5a59c709e254 admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:20.232013 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993909c0-80cf-4ebf-8dd9-5a59c709e254 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:20.234190 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993909c0-80cf-4ebf-8dd9-5a59c709e254 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:20.301943 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 827/1667] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:23:20 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:20.305861 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfbb431-81bb-4389-99d1-5dced4216f54 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:20.323825 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfbb431-81bb-4389-99d1-5dced4216f54 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:20.418350 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:20.491111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20779cc-24a1-4698-881b-048fb64a05ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:20.492066 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20779cc-24a1-4698-881b-048fb64a05ab admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:20.492529 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20779cc-24a1-4698-881b-048fb64a05ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:20.492933 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20779cc-24a1-4698-881b-048fb64a05ab admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:20.495002 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20779cc-24a1-4698-881b-048fb64a05ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:20.497796 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20779cc-24a1-4698-881b-048fb64a05ab admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:20.535713 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 841/1668] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:23:20 2022] POST /identity/v3/users => generated 294 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:20.541010 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993909c0-80cf-4ebf-8dd9-5a59c709e254 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:20.563455 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993909c0-80cf-4ebf-8dd9-5a59c709e254 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:20.587754 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bfee8e-7261-4fb2-aea8-a64a3f900ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:20.589490 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bfee8e-7261-4fb2-aea8-a64a3f900ebe admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:20.589490 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bfee8e-7261-4fb2-aea8-a64a3f900ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:20.589849 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bfee8e-7261-4fb2-aea8-a64a3f900ebe admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:20.591422 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bfee8e-7261-4fb2-aea8-a64a3f900ebe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:20.599528 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bfee8e-7261-4fb2-aea8-a64a3f900ebe admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:20.646171 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20779cc-24a1-4698-881b-048fb64a05ab admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:20.653450 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 828/1669] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:20 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:20.668207 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20779cc-24a1-4698-881b-048fb64a05ab admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:20.705891 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff421c9-462b-481b-a5b4-8892fba3b2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:20.706597 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff421c9-462b-481b-a5b4-8892fba3b2f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:20.709297 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff421c9-462b-481b-a5b4-8892fba3b2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:20.712981 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff421c9-462b-481b-a5b4-8892fba3b2f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:20.713904 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff421c9-462b-481b-a5b4-8892fba3b2f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:20.715622 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff421c9-462b-481b-a5b4-8892fba3b2f3 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:20.779334 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 842/1670] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:20 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:23:20.780646 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bfee8e-7261-4fb2-aea8-a64a3f900ebe admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:20.797320 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bfee8e-7261-4fb2-aea8-a64a3f900ebe admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:20.821343 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d564fe-0951-4e73-b6aa-c90ab565296e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:20.822353 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d564fe-0951-4e73-b6aa-c90ab565296e admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:20.823043 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d564fe-0951-4e73-b6aa-c90ab565296e admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:20.823323 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d564fe-0951-4e73-b6aa-c90ab565296e admin admin] PATH_INFO: `/v3/projects/4e53e203330b4867b34feafdedd69f24/users/32840f92964f4398a9c439d61ff10a08/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:20.863090 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:20.912143 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d564fe-0951-4e73-b6aa-c90ab565296e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e53e203330b4867b34feafdedd69f24, user_id=32840f92964f4398a9c439d61ff10a08, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:20.918804 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d564fe-0951-4e73-b6aa-c90ab565296e admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:20.972716 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 829/1671] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:23:20 2022] PUT /identity/v3/projects/4e53e203330b4867b34feafdedd69f24/users/32840f92964f4398a9c439d61ff10a08/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:20.984060 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91809867-5f80-4516-8f0d-951e5bb8e145 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:20.984601 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91809867-5f80-4516-8f0d-951e5bb8e145 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:20.985052 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91809867-5f80-4516-8f0d-951e5bb8e145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:21.078531 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-91809867-5f80-4516-8f0d-951e5bb8e145 None None] MFA Rules not processed for user `32840f92964f4398a9c439d61ff10a08`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:21.083152 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-91809867-5f80-4516-8f0d-951e5bb8e145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:21.138017 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d564fe-0951-4e73-b6aa-c90ab565296e admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:21.140202 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-92d564fe-0951-4e73-b6aa-c90ab565296e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:21.277717 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 843/1672] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:21.281904 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91809867-5f80-4516-8f0d-951e5bb8e145 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:21.368781 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d564fe-0951-4e73-b6aa-c90ab565296e admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:21.389880 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cb5c88-3e7e-4693-be36-c0250abf1d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:21.391757 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb5c88-3e7e-4693-be36-c0250abf1d5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:21.393811 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb5c88-3e7e-4693-be36-c0250abf1d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:21.395797 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cb5c88-3e7e-4693-be36-c0250abf1d5e admin admin] PATH_INFO: `/v3/projects/a99e18940a8943adaad20cdeadae1d1b` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:21.403166 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cb5c88-3e7e-4693-be36-c0250abf1d5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a99e18940a8943adaad20cdeadae1d1b)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:21.410678 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cb5c88-3e7e-4693-be36-c0250abf1d5e admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:21.566850 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91809867-5f80-4516-8f0d-951e5bb8e145 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:21.621704 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-b2cb5c88-3e7e-4693-be36-c0250abf1d5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a99e18940a8943adaad20cdeadae1d1b', 'request_id': 'req-b2cb5c88-3e7e-4693-be36-c0250abf1d5e'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:21.637712 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-b2cb5c88-3e7e-4693-be36-c0250abf1d5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a99e18940a8943adaad20cdeadae1d1b', 'request_id': 'req-b2cb5c88-3e7e-4693-be36-c0250abf1d5e'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:21.654878 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-b2cb5c88-3e7e-4693-be36-c0250abf1d5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a99e18940a8943adaad20cdeadae1d1b', 'request_id': 'req-b2cb5c88-3e7e-4693-be36-c0250abf1d5e'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:21.682524 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-b2cb5c88-3e7e-4693-be36-c0250abf1d5e admin admin] The token cache is being invalidate because project a99e18940a8943adaad20cdeadae1d1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:23:21.683186 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-b2cb5c88-3e7e-4693-be36-c0250abf1d5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:21.684620 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-b2cb5c88-3e7e-4693-be36-c0250abf1d5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:21.688027 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 830/1673] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:23:21 2022] DELETE /identity/v3/projects/a99e18940a8943adaad20cdeadae1d1b => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:21.692645 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db80f0-252b-4592-8d38-aa013857e875 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:21.693252 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db80f0-252b-4592-8d38-aa013857e875 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:21.693728 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70db80f0-252b-4592-8d38-aa013857e875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:21.713239 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:21.732372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf3d61b-0db2-4a95-96b0-ffc8c4fd0809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:21.733111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf3d61b-0db2-4a95-96b0-ffc8c4fd0809 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:21.733396 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf3d61b-0db2-4a95-96b0-ffc8c4fd0809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:21.733674 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf3d61b-0db2-4a95-96b0-ffc8c4fd0809 admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:21.734378 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf3d61b-0db2-4a95-96b0-ffc8c4fd0809 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:21.734692 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-70db80f0-252b-4592-8d38-aa013857e875 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:21.736156 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-70db80f0-252b-4592-8d38-aa013857e875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:21.736345 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf3d61b-0db2-4a95-96b0-ffc8c4fd0809 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:21.758474 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 831/1674] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:21.769628 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f5bc5-640d-434b-aa9e-f8ed9b39f6df None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:21.770040 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f5bc5-640d-434b-aa9e-f8ed9b39f6df None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:21.770541 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632f5bc5-640d-434b-aa9e-f8ed9b39f6df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:21.802867 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 844/1675] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:23:21 2022] POST /identity/v3/projects => generated 385 bytes in 521 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:23:21.804313 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-632f5bc5-640d-434b-aa9e-f8ed9b39f6df None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:21.805705 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-632f5bc5-640d-434b-aa9e-f8ed9b39f6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:21.808417 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf3d61b-0db2-4a95-96b0-ffc8c4fd0809 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:21.811144 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-0bf3d61b-0db2-4a95-96b0-ffc8c4fd0809 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:21.955812 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf3d61b-0db2-4a95-96b0-ffc8c4fd0809 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:21.970455 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0fc85f-0b4a-4061-adfb-8b0608cb93cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:21.971402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0fc85f-0b4a-4061-adfb-8b0608cb93cf admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:21.971822 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0fc85f-0b4a-4061-adfb-8b0608cb93cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:21.972229 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0fc85f-0b4a-4061-adfb-8b0608cb93cf admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:21.973137 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0fc85f-0b4a-4061-adfb-8b0608cb93cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:21.975244 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0fc85f-0b4a-4061-adfb-8b0608cb93cf admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:21.987717 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 832/1676] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:21.998510 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f5bc5-640d-434b-aa9e-f8ed9b39f6df None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:22.024688 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 845/1677] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:23:21 2022] POST /identity/v3/users => generated 299 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:23:22.029141 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0fc85f-0b4a-4061-adfb-8b0608cb93cf admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:22.034073 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632f5bc5-640d-434b-aa9e-f8ed9b39f6df None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:22.047908 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0fc85f-0b4a-4061-adfb-8b0608cb93cf admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:22.055644 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f7c992-ed0e-4d4a-9de2-e4e839728c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:22.058428 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7c992-ed0e-4d4a-9de2-e4e839728c93 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:22.060808 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7c992-ed0e-4d4a-9de2-e4e839728c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:22.061193 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70f7f64-ee7c-4df4-ae25-9a91c0eb8094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:22.061315 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7c992-ed0e-4d4a-9de2-e4e839728c93 admin admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:22.062768 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f7c992-ed0e-4d4a-9de2-e4e839728c93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:23:22.063194 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f7f64-ee7c-4df4-ae25-9a91c0eb8094 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:22.064143 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f7f64-ee7c-4df4-ae25-9a91c0eb8094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:22.064559 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f7f64-ee7c-4df4-ae25-9a91c0eb8094 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:22.064675 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f7c992-ed0e-4d4a-9de2-e4e839728c93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:22.065749 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70f7f64-ee7c-4df4-ae25-9a91c0eb8094 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:22.068360 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f7c992-ed0e-4d4a-9de2-e4e839728c93 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:22.070120 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70f7f64-ee7c-4df4-ae25-9a91c0eb8094 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:22.099300 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 846/1678] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:22 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:23:22.103577 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9664cf52-51f6-4ee0-9299-a70184197bfd None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:22.106767 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9664cf52-51f6-4ee0-9299-a70184197bfd None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:22.107256 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9664cf52-51f6-4ee0-9299-a70184197bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:22.110232 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:22.128922 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 833/1679] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:23:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:23:22.130042 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f7c992-ed0e-4d4a-9de2-e4e839728c93 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:22.141631 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-9664cf52-51f6-4ee0-9299-a70184197bfd None None] MFA Rules not processed for user `86a18e313e4944748aef58d54bfe7b13`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:22.142666 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-9664cf52-51f6-4ee0-9299-a70184197bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:22.156067 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f7c992-ed0e-4d4a-9de2-e4e839728c93 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:22.182836 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ec41c-86eb-43d2-b3d9-baa5c6167617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:22.188113 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ec41c-86eb-43d2-b3d9-baa5c6167617 admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:22.190053 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ec41c-86eb-43d2-b3d9-baa5c6167617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:22.191587 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ec41c-86eb-43d2-b3d9-baa5c6167617 admin admin] PATH_INFO: `/v3/projects/5160189da7cd4aa7b45a6fea816350c3/users/79235d82f24040d7bf932837332f58a9/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:22.217161 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 847/1680] 10.222.0.50 () {58 vars in 1009 bytes} [Fri May 27 03:23:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:22.223624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9664cf52-51f6-4ee0-9299-a70184197bfd None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:22.233506 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9664cf52-51f6-4ee0-9299-a70184197bfd None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:22.238510 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7583e23-5138-430c-84d7-da54ee427477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:22.239208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7583e23-5138-430c-84d7-da54ee427477 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:22.239486 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7583e23-5138-430c-84d7-da54ee427477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:22.239767 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7583e23-5138-430c-84d7-da54ee427477 admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:22.240467 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7583e23-5138-430c-84d7-da54ee427477 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:22.242920 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7583e23-5138-430c-84d7-da54ee427477 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:22.253190 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54ec41c-86eb-43d2-b3d9-baa5c6167617 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5160189da7cd4aa7b45a6fea816350c3, user_id=79235d82f24040d7bf932837332f58a9, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:22.262507 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54ec41c-86eb-43d2-b3d9-baa5c6167617 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:22.287714 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 848/1681] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:23:22 2022] POST /identity/v3/projects => generated 403 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:22.292404 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7583e23-5138-430c-84d7-da54ee427477 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:22.315204 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 834/1682] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:23:22 2022] PUT /identity/v3/projects/5160189da7cd4aa7b45a6fea816350c3/users/79235d82f24040d7bf932837332f58a9/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:22.319215 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ec41c-86eb-43d2-b3d9-baa5c6167617 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:22.433637 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:22.473966 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ec41c-86eb-43d2-b3d9-baa5c6167617 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:22.490885 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7583e23-5138-430c-84d7-da54ee427477 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:22.492388 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb41282-2c35-4443-9054-169c16d616fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:22.495318 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb41282-2c35-4443-9054-169c16d616fb admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:22.496042 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb41282-2c35-4443-9054-169c16d616fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:22.496326 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb41282-2c35-4443-9054-169c16d616fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:22.498735 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb41282-2c35-4443-9054-169c16d616fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:22.502934 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb41282-2c35-4443-9054-169c16d616fb admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:22.506075 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9350c-4362-417d-9a0a-5e1398e0ffa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:22.507249 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9350c-4362-417d-9a0a-5e1398e0ffa4 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:22.507534 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9350c-4362-417d-9a0a-5e1398e0ffa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:22.508479 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9350c-4362-417d-9a0a-5e1398e0ffa4 admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:22.509863 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde9350c-4362-417d-9a0a-5e1398e0ffa4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:22.515734 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde9350c-4362-417d-9a0a-5e1398e0ffa4 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:22.531795 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 835/1683] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:22 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:23:22.540211 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb41282-2c35-4443-9054-169c16d616fb admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:22.548789 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-6bb41282-2c35-4443-9054-169c16d616fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:22.588757 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 849/1684] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:23:22 2022] POST /identity/v3/users => generated 302 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:22.593984 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9350c-4362-417d-9a0a-5e1398e0ffa4 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:22.619192 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9350c-4362-417d-9a0a-5e1398e0ffa4 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:22.634409 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c3c1d0-c690-4d09-8381-7e35845a3db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:22.635538 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c3c1d0-c690-4d09-8381-7e35845a3db8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:22.637032 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c3c1d0-c690-4d09-8381-7e35845a3db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:22.637248 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c3c1d0-c690-4d09-8381-7e35845a3db8 admin admin] PATH_INFO: `/v3/projects/5160189da7cd4aa7b45a6fea816350c3/users/79235d82f24040d7bf932837332f58a9/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:22.643609 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c3c1d0-c690-4d09-8381-7e35845a3db8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5160189da7cd4aa7b45a6fea816350c3, user_id=79235d82f24040d7bf932837332f58a9, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:22.650407 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c3c1d0-c690-4d09-8381-7e35845a3db8 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:22.698489 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 850/1685] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:23:22 2022] PUT /identity/v3/projects/5160189da7cd4aa7b45a6fea816350c3/users/79235d82f24040d7bf932837332f58a9/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:22.710989 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c3c1d0-c690-4d09-8381-7e35845a3db8 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:22.793857 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb41282-2c35-4443-9054-169c16d616fb admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:22.915542 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c3c1d0-c690-4d09-8381-7e35845a3db8 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:22.931732 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee3195d-bf55-47ac-a6d2-14703823d86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:22.932486 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3195d-bf55-47ac-a6d2-14703823d86e admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:22.932812 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3195d-bf55-47ac-a6d2-14703823d86e admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:22.933061 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee3195d-bf55-47ac-a6d2-14703823d86e admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:22.933806 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee3195d-bf55-47ac-a6d2-14703823d86e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:22.935543 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee3195d-bf55-47ac-a6d2-14703823d86e admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:22.984392 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 851/1686] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:22 2022] GET /identity/v3/roles => generated 1420 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:22.985948 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926283a8-c0ec-491d-8a16-c6f1f4b27795 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:22.986237 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926283a8-c0ec-491d-8a16-c6f1f4b27795 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:22.986516 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926283a8-c0ec-491d-8a16-c6f1f4b27795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:23.019375 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f566e6-7d41-4a6d-8ef5-f89ef66d2481 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de53e2f0fba4e0fac70887067c1963d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:23.020063 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f566e6-7d41-4a6d-8ef5-f89ef66d2481 service swift] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:23.020382 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f566e6-7d41-4a6d-8ef5-f89ef66d2481 service swift] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:23.020668 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f566e6-7d41-4a6d-8ef5-f89ef66d2481 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:23.045116 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:23.055089 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-926283a8-c0ec-491d-8a16-c6f1f4b27795 None None] MFA Rules not processed for user `79235d82f24040d7bf932837332f58a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:23.058334 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-926283a8-c0ec-491d-8a16-c6f1f4b27795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:23.212088 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f566e6-7d41-4a6d-8ef5-f89ef66d2481 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:23.227647 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f566e6-7d41-4a6d-8ef5-f89ef66d2481 service swift] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:23.247598 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 836/1687] 10.222.0.50 () {60 vars in 1384 bytes} [Fri May 27 03:23:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 709 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:23:23.248700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f566e6-7d41-4a6d-8ef5-f89ef66d2481 service swift] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:23.269759 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f566e6-7d41-4a6d-8ef5-f89ef66d2481 service swift] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:23.283126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f2f689-0d7f-488c-a255-96a067d25482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:23.283832 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2f689-0d7f-488c-a255-96a067d25482 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:23.285051 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2f689-0d7f-488c-a255-96a067d25482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:23.285051 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2f689-0d7f-488c-a255-96a067d25482 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:23.285307 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f2f689-0d7f-488c-a255-96a067d25482 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:23.286749 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f2f689-0d7f-488c-a255-96a067d25482 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:23.318801 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 837/1688] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:23 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:23.320883 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f2f689-0d7f-488c-a255-96a067d25482 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:23.347416 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f2f689-0d7f-488c-a255-96a067d25482 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:23.347934 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 852/1689] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:22 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:23.349098 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926283a8-c0ec-491d-8a16-c6f1f4b27795 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:23.354015 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-926283a8-c0ec-491d-8a16-c6f1f4b27795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:23.360562 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb5480-8fe8-4f60-bf94-302da3a3598f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:23.361566 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb5480-8fe8-4f60-bf94-302da3a3598f admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:23.361855 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb5480-8fe8-4f60-bf94-302da3a3598f admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:23.362125 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb5480-8fe8-4f60-bf94-302da3a3598f admin admin] PATH_INFO: `/v3/projects/86f523489a104a25bbcd33215361c336/users/03af57e57529401ca3ce3b78ace5be3e/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:23.429508 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcb5480-8fe8-4f60-bf94-302da3a3598f admin admin] RBAC: Authorizing `identity:create_grant(project_id=86f523489a104a25bbcd33215361c336, user_id=03af57e57529401ca3ce3b78ace5be3e, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:23.432979 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcb5480-8fe8-4f60-bf94-302da3a3598f admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:23.482068 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 838/1690] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:23:23 2022] PUT /identity/v3/projects/86f523489a104a25bbcd33215361c336/users/03af57e57529401ca3ce3b78ace5be3e/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:23.487159 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9a10f-cddc-4ff5-9db8-9837117350ca None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:23.487623 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9a10f-cddc-4ff5-9db8-9837117350ca None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:23.488223 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc9a10f-cddc-4ff5-9db8-9837117350ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:23.508103 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:23.548968 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926283a8-c0ec-491d-8a16-c6f1f4b27795 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:23.572970 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-8fc9a10f-cddc-4ff5-9db8-9837117350ca None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:23.574262 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-8fc9a10f-cddc-4ff5-9db8-9837117350ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:23.687645 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15106d1e-502e-4b17-a581-44bcc297cd6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:23.688332 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15106d1e-502e-4b17-a581-44bcc297cd6f service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:23.688708 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15106d1e-502e-4b17-a581-44bcc297cd6f service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:23.689034 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15106d1e-502e-4b17-a581-44bcc297cd6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:23.859550 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15106d1e-502e-4b17-a581-44bcc297cd6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:23.864653 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15106d1e-502e-4b17-a581-44bcc297cd6f service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:23.895418 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 839/1691] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:23:23.901801 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc9a10f-cddc-4ff5-9db8-9837117350ca None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:23.904879 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-8fc9a10f-cddc-4ff5-9db8-9837117350ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:23.915776 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 853/1692] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:23 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:23.986099 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc9a10f-cddc-4ff5-9db8-9837117350ca None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:23.996104 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cfc482-e487-4d7b-9d32-9d3fb10cd455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:23.997049 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cfc482-e487-4d7b-9d32-9d3fb10cd455 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:23.997343 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cfc482-e487-4d7b-9d32-9d3fb10cd455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:23.997627 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cfc482-e487-4d7b-9d32-9d3fb10cd455 admin admin] PATH_INFO: `/v3/users/0731eb80ac2f4eae82a8aee6454c3840` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:23.999926 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cfc482-e487-4d7b-9d32-9d3fb10cd455 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0731eb80ac2f4eae82a8aee6454c3840)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:24.008528 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cfc482-e487-4d7b-9d32-9d3fb10cd455 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:24.062175 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15106d1e-502e-4b17-a581-44bcc297cd6f service cinder] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:24.086117 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15106d1e-502e-4b17-a581-44bcc297cd6f service cinder] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:24.100378 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb234c9-9864-4067-95c8-a96226aebb0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:24.101386 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb234c9-9864-4067-95c8-a96226aebb0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:24.101633 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb234c9-9864-4067-95c8-a96226aebb0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:24.101902 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb234c9-9864-4067-95c8-a96226aebb0c admin admin] PATH_INFO: `/v3/users/32840f92964f4398a9c439d61ff10a08` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:24.105672 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb234c9-9864-4067-95c8-a96226aebb0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=32840f92964f4398a9c439d61ff10a08)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:24.112912 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb234c9-9864-4067-95c8-a96226aebb0c admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:24.244815 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-7cb234c9-9864-4067-95c8-a96226aebb0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32840f92964f4398a9c439d61ff10a08'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:24.247592 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-7cb234c9-9864-4067-95c8-a96226aebb0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32840f92964f4398a9c439d61ff10a08'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:24.265216 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-7cb234c9-9864-4067-95c8-a96226aebb0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32840f92964f4398a9c439d61ff10a08'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:24.270536 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-7cb234c9-9864-4067-95c8-a96226aebb0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32840f92964f4398a9c439d61ff10a08'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:24.272413 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-a5cfc482-e487-4d7b-9d32-9d3fb10cd455 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0731eb80ac2f4eae82a8aee6454c3840'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:24.309176 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-7cb234c9-9864-4067-95c8-a96226aebb0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32840f92964f4398a9c439d61ff10a08'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:24.357757 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 854/1693] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:23:24 2022] DELETE /identity/v3/users/32840f92964f4398a9c439d61ff10a08 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:23:24.359219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb234c9-9864-4067-95c8-a96226aebb0c admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:24.361663 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-7cb234c9-9864-4067-95c8-a96226aebb0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:24.366960 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-a5cfc482-e487-4d7b-9d32-9d3fb10cd455 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0731eb80ac2f4eae82a8aee6454c3840'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:24.390116 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-a5cfc482-e487-4d7b-9d32-9d3fb10cd455 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0731eb80ac2f4eae82a8aee6454c3840'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:24.395633 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:24.412482 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-a5cfc482-e487-4d7b-9d32-9d3fb10cd455 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0731eb80ac2f4eae82a8aee6454c3840'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:24.417389 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:24.420940 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-a5cfc482-e487-4d7b-9d32-9d3fb10cd455 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0731eb80ac2f4eae82a8aee6454c3840'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:24.439340 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 840/1694] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:23:23 2022] DELETE /identity/v3/users/0731eb80ac2f4eae82a8aee6454c3840 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:24.439340 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af933de-5e34-451d-b365-0e2eacf9aaea None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:24.439340 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af933de-5e34-451d-b365-0e2eacf9aaea None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:24.439340 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af933de-5e34-451d-b365-0e2eacf9aaea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:24.466983 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-4af933de-5e34-451d-b365-0e2eacf9aaea None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:24.477035 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-4af933de-5e34-451d-b365-0e2eacf9aaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:24.521802 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb234c9-9864-4067-95c8-a96226aebb0c admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:24.707916 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 841/1695] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:24.714285 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7ce69-5047-465e-ba03-d6d05ad15f7c None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:24.715132 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7ce69-5047-465e-ba03-d6d05ad15f7c None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:24.716224 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff7ce69-5047-465e-ba03-d6d05ad15f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:24.721522 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb22997-5d74-4a76-b0d7-6a399bcb9b4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:24.724012 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb22997-5d74-4a76-b0d7-6a399bcb9b4c service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:24.726460 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb22997-5d74-4a76-b0d7-6a399bcb9b4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:24.727120 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb22997-5d74-4a76-b0d7-6a399bcb9b4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:24.736468 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-bcb22997-5d74-4a76-b0d7-6a399bcb9b4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:24.764741 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-dff7ce69-5047-465e-ba03-d6d05ad15f7c None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:24.767296 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-dff7ce69-5047-465e-ba03-d6d05ad15f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:24.767995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb22997-5d74-4a76-b0d7-6a399bcb9b4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:24.771720 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb22997-5d74-4a76-b0d7-6a399bcb9b4c service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:24.798036 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 855/1696] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:24.800009 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb22997-5d74-4a76-b0d7-6a399bcb9b4c service neutron] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:24.802083 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-bcb22997-5d74-4a76-b0d7-6a399bcb9b4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:24.815846 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb22997-5d74-4a76-b0d7-6a399bcb9b4c service neutron] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:24.821246 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 842/1697] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:24.824393 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e3a95-3fff-4897-a3ee-47c6cf841e48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:24.826026 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e3a95-3fff-4897-a3ee-47c6cf841e48 service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:24.826699 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e3a95-3fff-4897-a3ee-47c6cf841e48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:24.827943 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e3a95-3fff-4897-a3ee-47c6cf841e48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:24.853853 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e3a95-3fff-4897-a3ee-47c6cf841e48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:24.859899 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6e3a95-3fff-4897-a3ee-47c6cf841e48 service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:24.880714 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 856/1698] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:24.927482 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff7ce69-5047-465e-ba03-d6d05ad15f7c None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:24.946309 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff7ce69-5047-465e-ba03-d6d05ad15f7c None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:24.967983 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d019fb-e097-42ea-be7b-6fdac324e0f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:24.968837 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d019fb-e097-42ea-be7b-6fdac324e0f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:24.969947 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d019fb-e097-42ea-be7b-6fdac324e0f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:24.972998 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d019fb-e097-42ea-be7b-6fdac324e0f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:24.994674 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d019fb-e097-42ea-be7b-6fdac324e0f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:25.000572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d019fb-e097-42ea-be7b-6fdac324e0f7 service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:25.024166 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 843/1699] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:27.010718 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7d5e2-a92b-4151-8cda-631f6c5c5456 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:27.011319 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7d5e2-a92b-4151-8cda-631f6c5c5456 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:27.011319 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d7d5e2-a92b-4151-8cda-631f6c5c5456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:27.019012 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d019fb-e097-42ea-be7b-6fdac324e0f7 service neutron] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:27.021131 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-a3d019fb-e097-42ea-be7b-6fdac324e0f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:27.045671 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d019fb-e097-42ea-be7b-6fdac324e0f7 service neutron] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:27.065723 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-e4d7d5e2-a92b-4151-8cda-631f6c5c5456 None None] MFA Rules not processed for user `4ee30c0d6d694ee8945800157a9fc5b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:27.066900 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-e4d7d5e2-a92b-4151-8cda-631f6c5c5456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:27.068693 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d5ca9-001c-4ae7-8796-dbb679d8c5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:27.070218 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d5ca9-001c-4ae7-8796-dbb679d8c5cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:27.070829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d5ca9-001c-4ae7-8796-dbb679d8c5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:27.072415 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d5ca9-001c-4ae7-8796-dbb679d8c5cc admin admin] PATH_INFO: `/v3/projects/4e53e203330b4867b34feafdedd69f24` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:27.076916 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506d5ca9-001c-4ae7-8796-dbb679d8c5cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e53e203330b4867b34feafdedd69f24)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:27.086752 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506d5ca9-001c-4ae7-8796-dbb679d8c5cc admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:27.098861 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 857/1700] 10.222.0.67 () {58 vars in 1015 bytes} [Fri May 27 03:23:27 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:27.185918 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d7d5e2-a92b-4151-8cda-631f6c5c5456 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:27.194918 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d7d5e2-a92b-4151-8cda-631f6c5c5456 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:27.196906 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-506d5ca9-001c-4ae7-8796-dbb679d8c5cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e53e203330b4867b34feafdedd69f24', 'request_id': 'req-506d5ca9-001c-4ae7-8796-dbb679d8c5cc'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:27.214944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-506d5ca9-001c-4ae7-8796-dbb679d8c5cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e53e203330b4867b34feafdedd69f24', 'request_id': 'req-506d5ca9-001c-4ae7-8796-dbb679d8c5cc'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:27.231944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-506d5ca9-001c-4ae7-8796-dbb679d8c5cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e53e203330b4867b34feafdedd69f24', 'request_id': 'req-506d5ca9-001c-4ae7-8796-dbb679d8c5cc'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:27.266484 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-506d5ca9-001c-4ae7-8796-dbb679d8c5cc admin admin] The token cache is being invalidate because project 4e53e203330b4867b34feafdedd69f24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:23:27.266940 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-506d5ca9-001c-4ae7-8796-dbb679d8c5cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:27.275599 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-506d5ca9-001c-4ae7-8796-dbb679d8c5cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:27.278465 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 844/1701] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:23:27 2022] DELETE /identity/v3/projects/4e53e203330b4867b34feafdedd69f24 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:23:27.282589 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d5ca9-001c-4ae7-8796-dbb679d8c5cc admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:27.285922 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-506d5ca9-001c-4ae7-8796-dbb679d8c5cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:27.384307 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5024c8-015e-4ef6-a0e9-a66ab7454f44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:27.385352 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5024c8-015e-4ef6-a0e9-a66ab7454f44 service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:27.385783 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5024c8-015e-4ef6-a0e9-a66ab7454f44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:27.386199 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5024c8-015e-4ef6-a0e9-a66ab7454f44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:27.395315 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-fc5024c8-015e-4ef6-a0e9-a66ab7454f44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:27.414209 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:27.434181 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5024c8-015e-4ef6-a0e9-a66ab7454f44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:27.436138 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5024c8-015e-4ef6-a0e9-a66ab7454f44 service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:27.448157 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d5ca9-001c-4ae7-8796-dbb679d8c5cc admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:27.460897 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eab377-edc9-4fd2-8339-b9fb582ea271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:27.461224 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 858/1702] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:27.462315 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eab377-edc9-4fd2-8339-b9fb582ea271 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:27.462836 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eab377-edc9-4fd2-8339-b9fb582ea271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:27.463372 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eab377-edc9-4fd2-8339-b9fb582ea271 admin admin] PATH_INFO: `/v3/users/79235d82f24040d7bf932837332f58a9` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:27.464968 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eab377-edc9-4fd2-8339-b9fb582ea271 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79235d82f24040d7bf932837332f58a9)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:27.467525 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eab377-edc9-4fd2-8339-b9fb582ea271 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:27.588825 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-45eab377-edc9-4fd2-8339-b9fb582ea271 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79235d82f24040d7bf932837332f58a9'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:27.609775 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c9786-a3e2-4b03-ad70-861807c97720 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:27.610302 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c9786-a3e2-4b03-ad70-861807c97720 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:27.610738 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c9786-a3e2-4b03-ad70-861807c97720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:27.624419 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-45eab377-edc9-4fd2-8339-b9fb582ea271 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79235d82f24040d7bf932837332f58a9'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:27.651086 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-45eab377-edc9-4fd2-8339-b9fb582ea271 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79235d82f24040d7bf932837332f58a9'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:27.669749 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-45eab377-edc9-4fd2-8339-b9fb582ea271 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79235d82f24040d7bf932837332f58a9'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:27.669749 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-45eab377-edc9-4fd2-8339-b9fb582ea271 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79235d82f24040d7bf932837332f58a9'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:27.682642 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 845/1703] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:23:27 2022] DELETE /identity/v3/users/79235d82f24040d7bf932837332f58a9 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:27.711009 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-585c9786-a3e2-4b03-ad70-861807c97720 None None] MFA Rules not processed for user `468a46d55bd342739b9cd78bcd3a77fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:27.724469 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-585c9786-a3e2-4b03-ad70-861807c97720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:27.924479 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:27.998276 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 859/1704] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:28.004919 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce464583-0ed6-4461-ac81-88296d5d41af None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:28.005575 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce464583-0ed6-4461-ac81-88296d5d41af None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:28.005808 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce464583-0ed6-4461-ac81-88296d5d41af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:28.060844 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-ce464583-0ed6-4461-ac81-88296d5d41af None None] MFA Rules not processed for user `468a46d55bd342739b9cd78bcd3a77fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:28.062580 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-ce464583-0ed6-4461-ac81-88296d5d41af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:28.099370 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 846/1705] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:28.111102 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c9786-a3e2-4b03-ad70-861807c97720 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:28.121153 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-585c9786-a3e2-4b03-ad70-861807c97720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:28.429973 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c9786-a3e2-4b03-ad70-861807c97720 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:28.447199 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a227e644-1c0b-4b7d-92a2-3abd2722ff6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:28.449337 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a227e644-1c0b-4b7d-92a2-3abd2722ff6c service nova] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:28.449979 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a227e644-1c0b-4b7d-92a2-3abd2722ff6c service nova] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:28.450578 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a227e644-1c0b-4b7d-92a2-3abd2722ff6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:28.472355 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a227e644-1c0b-4b7d-92a2-3abd2722ff6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:28.479863 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a227e644-1c0b-4b7d-92a2-3abd2722ff6c service nova] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:28.544060 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 860/1706] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:28 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:30.214288 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce464583-0ed6-4461-ac81-88296d5d41af None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:30.215933 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-ce464583-0ed6-4461-ac81-88296d5d41af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:30.472341 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce464583-0ed6-4461-ac81-88296d5d41af None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:30.485494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5828c4ee-2c60-4f2c-a908-290b290db037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:30.486360 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5828c4ee-2c60-4f2c-a908-290b290db037 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:30.486727 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5828c4ee-2c60-4f2c-a908-290b290db037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:30.486982 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5828c4ee-2c60-4f2c-a908-290b290db037 admin admin] PATH_INFO: `/v3/projects/d1f5e434c7534753aea9c96091527cd1` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:30.489756 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5828c4ee-2c60-4f2c-a908-290b290db037 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1f5e434c7534753aea9c96091527cd1)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:30.494465 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5828c4ee-2c60-4f2c-a908-290b290db037 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:30.660041 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-5828c4ee-2c60-4f2c-a908-290b290db037 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1f5e434c7534753aea9c96091527cd1', 'request_id': 'req-5828c4ee-2c60-4f2c-a908-290b290db037'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:30.681844 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-5828c4ee-2c60-4f2c-a908-290b290db037 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1f5e434c7534753aea9c96091527cd1', 'request_id': 'req-5828c4ee-2c60-4f2c-a908-290b290db037'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:30.717834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-5828c4ee-2c60-4f2c-a908-290b290db037 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1f5e434c7534753aea9c96091527cd1', 'request_id': 'req-5828c4ee-2c60-4f2c-a908-290b290db037'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:30.736133 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:30.769951 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-5828c4ee-2c60-4f2c-a908-290b290db037 admin admin] The token cache is being invalidate because project d1f5e434c7534753aea9c96091527cd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:23:30.770242 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-5828c4ee-2c60-4f2c-a908-290b290db037 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:30.773581 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-5828c4ee-2c60-4f2c-a908-290b290db037 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:30.779455 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 847/1707] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:23:30 2022] DELETE /identity/v3/projects/d1f5e434c7534753aea9c96091527cd1 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:30.784711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af04f49f-fbda-4106-bcde-16bbf7623a01 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:30.785675 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af04f49f-fbda-4106-bcde-16bbf7623a01 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:30.786157 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af04f49f-fbda-4106-bcde-16bbf7623a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:30.863136 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-af04f49f-fbda-4106-bcde-16bbf7623a01 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:30.864690 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-af04f49f-fbda-4106-bcde-16bbf7623a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:31.163799 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 861/1708] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:31.174030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a924b-327b-49ba-b6e2-05c57649a519 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:31.174596 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a924b-327b-49ba-b6e2-05c57649a519 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:31.175288 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a924b-327b-49ba-b6e2-05c57649a519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:31.236456 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-8d9a924b-327b-49ba-b6e2-05c57649a519 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:31.241193 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-8d9a924b-327b-49ba-b6e2-05c57649a519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:31.286017 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 848/1709] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:31.289679 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af04f49f-fbda-4106-bcde-16bbf7623a01 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:31.308904 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af04f49f-fbda-4106-bcde-16bbf7623a01 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:31.331759 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68174fc1-264b-4a42-8678-bb707e672846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:31.332563 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68174fc1-264b-4a42-8678-bb707e672846 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:31.332985 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68174fc1-264b-4a42-8678-bb707e672846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:31.333384 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68174fc1-264b-4a42-8678-bb707e672846 admin admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:31.334238 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68174fc1-264b-4a42-8678-bb707e672846 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:23:31.334634 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68174fc1-264b-4a42-8678-bb707e672846 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:31.338836 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68174fc1-264b-4a42-8678-bb707e672846 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:31.417831 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 862/1710] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:23:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:23:31.422239 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a924b-327b-49ba-b6e2-05c57649a519 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:31.434018 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a924b-327b-49ba-b6e2-05c57649a519 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:31.454486 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd373e-9af3-4c88-b7cc-41201658c9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:31.455468 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd373e-9af3-4c88-b7cc-41201658c9da admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:31.455893 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd373e-9af3-4c88-b7cc-41201658c9da admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:31.456304 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd373e-9af3-4c88-b7cc-41201658c9da admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:31.457208 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cd373e-9af3-4c88-b7cc-41201658c9da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:31.460027 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cd373e-9af3-4c88-b7cc-41201658c9da admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:31.545472 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 849/1711] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:23:31 2022] POST /identity/v3/projects => generated 373 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:31.549576 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68174fc1-264b-4a42-8678-bb707e672846 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:31.564517 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68174fc1-264b-4a42-8678-bb707e672846 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:31.789198 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27401a3-74f4-4fc1-a05d-0b86a2c3d9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:31.789966 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27401a3-74f4-4fc1-a05d-0b86a2c3d9ec admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:31.790610 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27401a3-74f4-4fc1-a05d-0b86a2c3d9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:31.790988 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27401a3-74f4-4fc1-a05d-0b86a2c3d9ec admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:31.793471 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27401a3-74f4-4fc1-a05d-0b86a2c3d9ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:31.797027 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27401a3-74f4-4fc1-a05d-0b86a2c3d9ec admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:31.836360 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:31.871453 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 863/1712] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:23:31 2022] POST /identity/v3/users => generated 287 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:31.874885 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd373e-9af3-4c88-b7cc-41201658c9da admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:31.899096 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd373e-9af3-4c88-b7cc-41201658c9da admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:31.909591 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a694d2-34fa-4b9c-a22c-2e80aeefa8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:31.910340 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a694d2-34fa-4b9c-a22c-2e80aeefa8b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:31.910691 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a694d2-34fa-4b9c-a22c-2e80aeefa8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:31.911120 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a694d2-34fa-4b9c-a22c-2e80aeefa8b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:31.912127 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a694d2-34fa-4b9c-a22c-2e80aeefa8b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:31.914038 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a694d2-34fa-4b9c-a22c-2e80aeefa8b1 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:31.930736 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 850/1713] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:31.932256 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27401a3-74f4-4fc1-a05d-0b86a2c3d9ec admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:31.963442 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27401a3-74f4-4fc1-a05d-0b86a2c3d9ec admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:31.972129 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb6523c-f2cf-4e08-8d38-d4dee41e46f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:31.972976 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb6523c-f2cf-4e08-8d38-d4dee41e46f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:31.973389 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb6523c-f2cf-4e08-8d38-d4dee41e46f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:31.973869 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb6523c-f2cf-4e08-8d38-d4dee41e46f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:31.974718 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb6523c-f2cf-4e08-8d38-d4dee41e46f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:31.976784 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb6523c-f2cf-4e08-8d38-d4dee41e46f2 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:31.998678 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 864/1714] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:32.003431 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a694d2-34fa-4b9c-a22c-2e80aeefa8b1 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:32.020309 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b889a-6fe5-4152-afd4-7099e85b2e1b None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:32.020762 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b889a-6fe5-4152-afd4-7099e85b2e1b None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:32.021211 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11b889a-6fe5-4152-afd4-7099e85b2e1b None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:32.033979 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 865/1715] 10.222.0.50 () {54 vars in 912 bytes} [Fri May 27 03:23:32 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:23:32.036629 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a694d2-34fa-4b9c-a22c-2e80aeefa8b1 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:32.044399 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb3c3ec-24f2-4536-88f9-6addb11b98eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:32.047564 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea6785-305b-4dac-aa33-0cae27523b5e None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:32.048215 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea6785-305b-4dac-aa33-0cae27523b5e None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:32.049069 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea6785-305b-4dac-aa33-0cae27523b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:32.055792 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3c3ec-24f2-4536-88f9-6addb11b98eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:32.056784 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3c3ec-24f2-4536-88f9-6addb11b98eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:32.059303 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb3c3ec-24f2-4536-88f9-6addb11b98eb admin admin] PATH_INFO: `/v3/projects/f7a93f7132614f2093ab91d2b205f4e9/users/2add2be353aa46d5ab7e6c02b26f51f7/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:32.119946 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb3c3ec-24f2-4536-88f9-6addb11b98eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7a93f7132614f2093ab91d2b205f4e9, user_id=2add2be353aa46d5ab7e6c02b26f51f7, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:32.123241 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb3c3ec-24f2-4536-88f9-6addb11b98eb admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:32.129519 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-f6ea6785-305b-4dac-aa33-0cae27523b5e None None] MFA Rules not processed for user `4ee30c0d6d694ee8945800157a9fc5b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:32.133846 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-f6ea6785-305b-4dac-aa33-0cae27523b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:32.162271 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 851/1716] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:23:32 2022] PUT /identity/v3/projects/f7a93f7132614f2093ab91d2b205f4e9/users/2add2be353aa46d5ab7e6c02b26f51f7/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:32.174633 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc69ffb-0f8e-42a0-b14e-e1543000ce09 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:32.175048 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc69ffb-0f8e-42a0-b14e-e1543000ce09 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:32.175454 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc69ffb-0f8e-42a0-b14e-e1543000ce09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:32.311574 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-ccc69ffb-0f8e-42a0-b14e-e1543000ce09 None None] MFA Rules not processed for user `2add2be353aa46d5ab7e6c02b26f51f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:32.312827 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-ccc69ffb-0f8e-42a0-b14e-e1543000ce09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:32.364820 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 866/1717] 10.222.0.50 () {58 vars in 1009 bytes} [Fri May 27 03:23:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:23:32.464684 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea6785-305b-4dac-aa33-0cae27523b5e None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:32.466306 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-f6ea6785-305b-4dac-aa33-0cae27523b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:32.484775 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea6785-305b-4dac-aa33-0cae27523b5e None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:32.532490 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 852/1718] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:32.534804 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc69ffb-0f8e-42a0-b14e-e1543000ce09 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:32.539398 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-ccc69ffb-0f8e-42a0-b14e-e1543000ce09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:32.733280 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005292de-26cb-4f4a-baf1-f33133848515 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:32.734218 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005292de-26cb-4f4a-baf1-f33133848515 service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:32.734596 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005292de-26cb-4f4a-baf1-f33133848515 service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:32.735267 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005292de-26cb-4f4a-baf1-f33133848515 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:32.769388 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005292de-26cb-4f4a-baf1-f33133848515 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:32.771108 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005292de-26cb-4f4a-baf1-f33133848515 service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:32.798084 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 867/1719] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:32.802520 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf39204-8e76-437f-bf72-b7bbbdcccf70 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:32.802881 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf39204-8e76-437f-bf72-b7bbbdcccf70 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:32.803247 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf39204-8e76-437f-bf72-b7bbbdcccf70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:32.818233 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc69ffb-0f8e-42a0-b14e-e1543000ce09 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:32.836336 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12af5a4-9636-4ca2-9daf-d221926a4ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:32.838212 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12af5a4-9636-4ca2-9daf-d221926a4ee5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:32.838502 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12af5a4-9636-4ca2-9daf-d221926a4ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:32.838969 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12af5a4-9636-4ca2-9daf-d221926a4ee5 admin admin] PATH_INFO: `/v3/projects/5160189da7cd4aa7b45a6fea816350c3` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:32.869706 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12af5a4-9636-4ca2-9daf-d221926a4ee5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5160189da7cd4aa7b45a6fea816350c3)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:32.879433 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-bdf39204-8e76-437f-bf72-b7bbbdcccf70 None None] MFA Rules not processed for user `03af57e57529401ca3ce3b78ace5be3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:32.880901 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-bdf39204-8e76-437f-bf72-b7bbbdcccf70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:32.882629 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12af5a4-9636-4ca2-9daf-d221926a4ee5 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:32.989870 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:33.080788 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-e12af5a4-9636-4ca2-9daf-d221926a4ee5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5160189da7cd4aa7b45a6fea816350c3', 'request_id': 'req-e12af5a4-9636-4ca2-9daf-d221926a4ee5'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:33.101045 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-e12af5a4-9636-4ca2-9daf-d221926a4ee5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5160189da7cd4aa7b45a6fea816350c3', 'request_id': 'req-e12af5a4-9636-4ca2-9daf-d221926a4ee5'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:33.124474 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-e12af5a4-9636-4ca2-9daf-d221926a4ee5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5160189da7cd4aa7b45a6fea816350c3', 'request_id': 'req-e12af5a4-9636-4ca2-9daf-d221926a4ee5'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:33.156722 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-e12af5a4-9636-4ca2-9daf-d221926a4ee5 admin admin] The token cache is being invalidate because project 5160189da7cd4aa7b45a6fea816350c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:23:33.157565 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-e12af5a4-9636-4ca2-9daf-d221926a4ee5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:33.158394 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-e12af5a4-9636-4ca2-9daf-d221926a4ee5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:33.163088 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 853/1720] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:23:32 2022] DELETE /identity/v3/projects/5160189da7cd4aa7b45a6fea816350c3 => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:33.173372 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05811aed-bc8a-4f0a-af78-6624c25d93ce None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:33.173776 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05811aed-bc8a-4f0a-af78-6624c25d93ce None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:33.174120 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05811aed-bc8a-4f0a-af78-6624c25d93ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:33.176652 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 868/1721] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:32 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:33.182044 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf39204-8e76-437f-bf72-b7bbbdcccf70 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:33.184259 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-bdf39204-8e76-437f-bf72-b7bbbdcccf70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:33.219029 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf39204-8e76-437f-bf72-b7bbbdcccf70 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:33.227002 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-05811aed-bc8a-4f0a-af78-6624c25d93ce None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:33.230029 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-05811aed-bc8a-4f0a-af78-6624c25d93ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:33.273710 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:33.426965 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 854/1722] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:33.430825 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05811aed-bc8a-4f0a-af78-6624c25d93ce None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:33.434105 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-05811aed-bc8a-4f0a-af78-6624c25d93ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:33.447483 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3ec701-76f3-468c-b37e-01300ee3b5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:33.448992 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ec701-76f3-468c-b37e-01300ee3b5a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:33.449483 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ec701-76f3-468c-b37e-01300ee3b5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:33.449986 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3ec701-76f3-468c-b37e-01300ee3b5a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:33.450895 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3ec701-76f3-468c-b37e-01300ee3b5a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:33.453567 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3ec701-76f3-468c-b37e-01300ee3b5a7 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:33.510525 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 869/1723] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:23:33 2022] POST /identity/v3/projects => generated 405 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:33.512684 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c42bfd-a856-412d-b15e-07f3b59f2292 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:33.513275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c42bfd-a856-412d-b15e-07f3b59f2292 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:33.513775 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c42bfd-a856-412d-b15e-07f3b59f2292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:33.579854 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-98c42bfd-a856-412d-b15e-07f3b59f2292 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:33.582758 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-98c42bfd-a856-412d-b15e-07f3b59f2292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:33.653125 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05811aed-bc8a-4f0a-af78-6624c25d93ce None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:33.796094 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:33.815353 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 870/1724] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:33.816655 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c42bfd-a856-412d-b15e-07f3b59f2292 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:33.818791 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-98c42bfd-a856-412d-b15e-07f3b59f2292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:33.839618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cef48e-7bf9-4249-9972-c3f214fbfd1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:33.840449 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cef48e-7bf9-4249-9972-c3f214fbfd1c service nova] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:33.840614 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cef48e-7bf9-4249-9972-c3f214fbfd1c service nova] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:33.840891 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cef48e-7bf9-4249-9972-c3f214fbfd1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:33.841319 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c42bfd-a856-412d-b15e-07f3b59f2292 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:33.848228 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-16cef48e-7bf9-4249-9972-c3f214fbfd1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:33.851178 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475cb9a-6d38-4dc5-aac1-d87485784bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:33.852044 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475cb9a-6d38-4dc5-aac1-d87485784bc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:33.852512 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475cb9a-6d38-4dc5-aac1-d87485784bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:33.852888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475cb9a-6d38-4dc5-aac1-d87485784bc4 admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:33.853717 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1475cb9a-6d38-4dc5-aac1-d87485784bc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:33.857504 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1475cb9a-6d38-4dc5-aac1-d87485784bc4 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:33.916342 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 871/1725] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:23:33 2022] POST /identity/v3/users => generated 309 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:33.917450 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475cb9a-6d38-4dc5-aac1-d87485784bc4 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:33.932835 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475cb9a-6d38-4dc5-aac1-d87485784bc4 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:33.941068 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc0361-acf5-4f88-a4e0-147bbaadf9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:33.942213 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc0361-acf5-4f88-a4e0-147bbaadf9fd admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:33.943195 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc0361-acf5-4f88-a4e0-147bbaadf9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:33.943594 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc0361-acf5-4f88-a4e0-147bbaadf9fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:33.944459 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cc0361-acf5-4f88-a4e0-147bbaadf9fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:23:33.944871 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cc0361-acf5-4f88-a4e0-147bbaadf9fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:33.947756 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cc0361-acf5-4f88-a4e0-147bbaadf9fd admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:33.965448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 872/1726] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:23:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:23:33.967041 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c464293-30d6-4877-9b19-45003682d364 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:33.967435 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c464293-30d6-4877-9b19-45003682d364 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:33.967811 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c464293-30d6-4877-9b19-45003682d364 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:33.969554 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 873/1727] 10.222.0.50 () {54 vars in 921 bytes} [Fri May 27 03:23:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:23:33.971375 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c464293-30d6-4877-9b19-45003682d364 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:33.986738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cef48e-7bf9-4249-9972-c3f214fbfd1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:33.988428 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cef48e-7bf9-4249-9972-c3f214fbfd1c service nova] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:33.995052 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c464293-30d6-4877-9b19-45003682d364 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:34.009234 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96919716-7d57-4b19-b1d3-a9cf241a31d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:34.010142 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96919716-7d57-4b19-b1d3-a9cf241a31d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:34.010527 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96919716-7d57-4b19-b1d3-a9cf241a31d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:34.010938 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96919716-7d57-4b19-b1d3-a9cf241a31d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:34.013181 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96919716-7d57-4b19-b1d3-a9cf241a31d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:34.014819 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 855/1728] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:33 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:34.019246 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96919716-7d57-4b19-b1d3-a9cf241a31d1 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:34.022270 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cef48e-7bf9-4249-9972-c3f214fbfd1c service nova] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:34.044276 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 874/1729] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:33 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:34.049537 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96919716-7d57-4b19-b1d3-a9cf241a31d1 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:34.074161 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cef48e-7bf9-4249-9972-c3f214fbfd1c service nova] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:34.078827 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96919716-7d57-4b19-b1d3-a9cf241a31d1 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:34.093247 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e81e10-408d-4463-9ea7-823661e86842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:34.094397 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e81e10-408d-4463-9ea7-823661e86842 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:34.095276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f3f6f-584a-453e-ae1f-fec5e0addbf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:34.096264 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e81e10-408d-4463-9ea7-823661e86842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:34.096559 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e81e10-408d-4463-9ea7-823661e86842 admin admin] PATH_INFO: `/v3/projects/6263657281eb42e691c38e6a90f57a9d/users/053412d72e6e4b4e8865d50a1594a3b4/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:34.100245 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f3f6f-584a-453e-ae1f-fec5e0addbf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:34.104145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f3f6f-584a-453e-ae1f-fec5e0addbf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:34.104928 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956f3f6f-584a-453e-ae1f-fec5e0addbf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:34.106018 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f3f6f-584a-453e-ae1f-fec5e0addbf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:34.115410 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956f3f6f-584a-453e-ae1f-fec5e0addbf3 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:34.135559 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e81e10-408d-4463-9ea7-823661e86842 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6263657281eb42e691c38e6a90f57a9d, user_id=053412d72e6e4b4e8865d50a1594a3b4, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:34.145264 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e81e10-408d-4463-9ea7-823661e86842 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:34.177243 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 875/1730] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:23:34 2022] PUT /identity/v3/projects/6263657281eb42e691c38e6a90f57a9d/users/053412d72e6e4b4e8865d50a1594a3b4/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:34.177243 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20827d-de6f-411d-b0db-1f9b2cb80c2e None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:34.179434 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20827d-de6f-411d-b0db-1f9b2cb80c2e None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:34.179724 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20827d-de6f-411d-b0db-1f9b2cb80c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:34.226614 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 856/1731] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:23:34 2022] POST /identity/v3/projects => generated 375 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:34.228189 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f3f6f-584a-453e-ae1f-fec5e0addbf3 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:34.231441 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-7d20827d-de6f-411d-b0db-1f9b2cb80c2e None None] MFA Rules not processed for user `1a699988f08a4001b3b7d764f01f4fc5`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:34.233778 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-7d20827d-de6f-411d-b0db-1f9b2cb80c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:34.331027 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:34.425097 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 876/1732] 10.222.0.50 () {58 vars in 1018 bytes} [Fri May 27 03:23:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:34.427978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20827d-de6f-411d-b0db-1f9b2cb80c2e None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:34.472435 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956f3f6f-584a-453e-ae1f-fec5e0addbf3 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:34.480728 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924485b7-b028-4649-af7c-852aad348a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:34.481664 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924485b7-b028-4649-af7c-852aad348a0e admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:34.482042 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924485b7-b028-4649-af7c-852aad348a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:34.482525 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924485b7-b028-4649-af7c-852aad348a0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:34.483276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924485b7-b028-4649-af7c-852aad348a0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:34.486928 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924485b7-b028-4649-af7c-852aad348a0e admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:34.517856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 857/1733] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:34 2022] GET /identity/v3/roles => generated 1420 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:34.518949 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924485b7-b028-4649-af7c-852aad348a0e admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:34.542358 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924485b7-b028-4649-af7c-852aad348a0e admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:34.560698 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fdd7b3-b028-4829-8ddf-c25bfe3cec9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:34.563138 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fdd7b3-b028-4829-8ddf-c25bfe3cec9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:34.565576 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fdd7b3-b028-4829-8ddf-c25bfe3cec9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:34.568792 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fdd7b3-b028-4829-8ddf-c25bfe3cec9c admin admin] PATH_INFO: `/v3/projects/6263657281eb42e691c38e6a90f57a9d/users/053412d72e6e4b4e8865d50a1594a3b4/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:34.582382 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fdd7b3-b028-4829-8ddf-c25bfe3cec9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6263657281eb42e691c38e6a90f57a9d, user_id=053412d72e6e4b4e8865d50a1594a3b4, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:34.591562 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fdd7b3-b028-4829-8ddf-c25bfe3cec9c admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:34.643324 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 858/1734] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:23:34 2022] PUT /identity/v3/projects/6263657281eb42e691c38e6a90f57a9d/users/053412d72e6e4b4e8865d50a1594a3b4/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:34.680819 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d20827d-de6f-411d-b0db-1f9b2cb80c2e None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:34.758017 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798199f0-8797-42d5-a885-c979b346d4f8 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:34.758586 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798199f0-8797-42d5-a885-c979b346d4f8 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:34.759935 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798199f0-8797-42d5-a885-c979b346d4f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:34.876485 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-798199f0-8797-42d5-a885-c979b346d4f8 None None] MFA Rules not processed for user `19457c759f0b4f5cbc530364af1f47d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:34.881070 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-798199f0-8797-42d5-a885-c979b346d4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:34.904273 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bcc585-4cd4-4181-9381-2f8491cfa635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:34.905041 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bcc585-4cd4-4181-9381-2f8491cfa635 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:34.905299 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bcc585-4cd4-4181-9381-2f8491cfa635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:34.905574 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bcc585-4cd4-4181-9381-2f8491cfa635 admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:34.906599 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bcc585-4cd4-4181-9381-2f8491cfa635 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:34.908270 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bcc585-4cd4-4181-9381-2f8491cfa635 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:35.116699 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 877/1735] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:23:34 2022] POST /identity/v3/users => generated 288 bytes in 691 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:35.122419 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bcc585-4cd4-4181-9381-2f8491cfa635 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:35.170571 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:35.209653 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bcc585-4cd4-4181-9381-2f8491cfa635 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:35.245221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9785eb0d-a7e3-46ae-a712-93d995c943c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:35.246209 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785eb0d-a7e3-46ae-a712-93d995c943c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:35.246640 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785eb0d-a7e3-46ae-a712-93d995c943c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:35.247058 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785eb0d-a7e3-46ae-a712-93d995c943c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:35.247953 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9785eb0d-a7e3-46ae-a712-93d995c943c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:35.249798 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9785eb0d-a7e3-46ae-a712-93d995c943c9 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:35.315893 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 859/1736] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 561 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:35.323198 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 878/1737] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:35 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:35.335019 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798199f0-8797-42d5-a885-c979b346d4f8 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:35.359678 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798199f0-8797-42d5-a885-c979b346d4f8 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:35.407525 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981cb27-7bdd-45f1-a2a9-48acbf833eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:35.411451 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981cb27-7bdd-45f1-a2a9-48acbf833eb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:35.414634 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981cb27-7bdd-45f1-a2a9-48acbf833eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:35.418973 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981cb27-7bdd-45f1-a2a9-48acbf833eb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:35.425493 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1981cb27-7bdd-45f1-a2a9-48acbf833eb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:35.432333 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1981cb27-7bdd-45f1-a2a9-48acbf833eb6 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:35.488934 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9785eb0d-a7e3-46ae-a712-93d995c943c9 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:35.491942 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 860/1738] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:35 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:35.502463 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981cb27-7bdd-45f1-a2a9-48acbf833eb6 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:35.550450 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1981cb27-7bdd-45f1-a2a9-48acbf833eb6 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:35.567340 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbe19b2-f4c0-4d9a-9ddd-45c863699901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:35.568303 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe19b2-f4c0-4d9a-9ddd-45c863699901 admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:35.568582 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe19b2-f4c0-4d9a-9ddd-45c863699901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:35.568853 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe19b2-f4c0-4d9a-9ddd-45c863699901 admin admin] PATH_INFO: `/v3/projects/bce58ee1ad2f4ac4a0afbfd819d2cea0/users/3be0341c558f4e40b1e5b909c79a52ca/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:35.606733 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbe19b2-f4c0-4d9a-9ddd-45c863699901 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bce58ee1ad2f4ac4a0afbfd819d2cea0, user_id=3be0341c558f4e40b1e5b909c79a52ca, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:35.609162 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbe19b2-f4c0-4d9a-9ddd-45c863699901 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:35.628476 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 861/1739] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:23:35 2022] PUT /identity/v3/projects/bce58ee1ad2f4ac4a0afbfd819d2cea0/users/3be0341c558f4e40b1e5b909c79a52ca/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:35.636566 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:35.641169 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a98dab-7a32-4828-94cd-8c674f5adfef None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:35.641169 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a98dab-7a32-4828-94cd-8c674f5adfef None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:35.641169 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a98dab-7a32-4828-94cd-8c674f5adfef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:35.664293 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9785eb0d-a7e3-46ae-a712-93d995c943c9 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:35.721511 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-81a98dab-7a32-4828-94cd-8c674f5adfef None None] MFA Rules not processed for user `3be0341c558f4e40b1e5b909c79a52ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:35.724418 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-81a98dab-7a32-4828-94cd-8c674f5adfef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:35.776732 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e5fa61-3137-4b1d-a54e-533648e8781e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:35.777464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5fa61-3137-4b1d-a54e-533648e8781e service nova] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:35.777793 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5fa61-3137-4b1d-a54e-533648e8781e service nova] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:35.778068 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e5fa61-3137-4b1d-a54e-533648e8781e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:35.792318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e5fa61-3137-4b1d-a54e-533648e8781e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:35.798781 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e5fa61-3137-4b1d-a54e-533648e8781e service nova] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:35.817512 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 879/1740] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:35.999625 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 862/1741] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:36.011281 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e5fa61-3137-4b1d-a54e-533648e8781e service nova] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:36.013717 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-06e5fa61-3137-4b1d-a54e-533648e8781e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:36.266348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3dc8c3-6fdd-4054-9b3e-c25639d661b2 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:36.266634 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3dc8c3-6fdd-4054-9b3e-c25639d661b2 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:36.266914 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3dc8c3-6fdd-4054-9b3e-c25639d661b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:36.299747 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e5fa61-3137-4b1d-a54e-533648e8781e service nova] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:36.319349 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a60cbe-5c26-469b-a6f1-d95f1d38d6f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:36.320863 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-ff3dc8c3-6fdd-4054-9b3e-c25639d661b2 None None] MFA Rules not processed for user `468a46d55bd342739b9cd78bcd3a77fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:36.322448 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-ff3dc8c3-6fdd-4054-9b3e-c25639d661b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:36.326727 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a60cbe-5c26-469b-a6f1-d95f1d38d6f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:36.327012 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a60cbe-5c26-469b-a6f1-d95f1d38d6f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:36.327300 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a60cbe-5c26-469b-a6f1-d95f1d38d6f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:36.375822 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a60cbe-5c26-469b-a6f1-d95f1d38d6f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:36.377366 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a60cbe-5c26-469b-a6f1-d95f1d38d6f7 service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:36.431953 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 880/1742] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:36.431953 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a60cbe-5c26-469b-a6f1-d95f1d38d6f7 service cinder] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:36.484656 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a60cbe-5c26-469b-a6f1-d95f1d38d6f7 service cinder] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:36.529221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54326aec-b596-4a3f-aadc-c1f9fbc25a3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:36.529882 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54326aec-b596-4a3f-aadc-c1f9fbc25a3c service nova] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:36.530185 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54326aec-b596-4a3f-aadc-c1f9fbc25a3c service nova] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:36.531531 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54326aec-b596-4a3f-aadc-c1f9fbc25a3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:36.587996 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 863/1743] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:36.693478 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54326aec-b596-4a3f-aadc-c1f9fbc25a3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:36.698510 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54326aec-b596-4a3f-aadc-c1f9fbc25a3c service nova] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:36.727581 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 881/1744] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:36.988741 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a295356b-04b3-4e2c-8163-36d569f9310b None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:36.989825 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a295356b-04b3-4e2c-8163-36d569f9310b None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:36.991704 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a295356b-04b3-4e2c-8163-36d569f9310b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:37.114941 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-a295356b-04b3-4e2c-8163-36d569f9310b None None] MFA Rules not processed for user `dd9938d83dc84ae494310741e2cba2ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:37.117657 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-a295356b-04b3-4e2c-8163-36d569f9310b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:37.214176 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:37.338319 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 864/1745] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:37.485970 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54326aec-b596-4a3f-aadc-c1f9fbc25a3c service nova] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:37.511558 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54326aec-b596-4a3f-aadc-c1f9fbc25a3c service nova] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:37.524602 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba41c17-9094-4338-9196-b89ffc1c461a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:37.525427 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba41c17-9094-4338-9196-b89ffc1c461a service nova] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:37.526379 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba41c17-9094-4338-9196-b89ffc1c461a service nova] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:37.527271 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba41c17-9094-4338-9196-b89ffc1c461a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:37.551095 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba41c17-9094-4338-9196-b89ffc1c461a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:37.553207 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba41c17-9094-4338-9196-b89ffc1c461a service nova] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:37.592990 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 882/1746] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:38.554815 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a295356b-04b3-4e2c-8163-36d569f9310b None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:38.557219 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-a295356b-04b3-4e2c-8163-36d569f9310b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:38.739936 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a295356b-04b3-4e2c-8163-36d569f9310b None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:38.759896 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88729a1c-515d-49bd-9464-c6afc5f039e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:38.760922 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88729a1c-515d-49bd-9464-c6afc5f039e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:38.761294 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88729a1c-515d-49bd-9464-c6afc5f039e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:38.761637 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88729a1c-515d-49bd-9464-c6afc5f039e6 admin admin] PATH_INFO: `/v3/users/2add2be353aa46d5ab7e6c02b26f51f7` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:38.765581 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88729a1c-515d-49bd-9464-c6afc5f039e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2add2be353aa46d5ab7e6c02b26f51f7)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:38.778862 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88729a1c-515d-49bd-9464-c6afc5f039e6 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:38.901957 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-88729a1c-515d-49bd-9464-c6afc5f039e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2add2be353aa46d5ab7e6c02b26f51f7'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:38.928509 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-88729a1c-515d-49bd-9464-c6afc5f039e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2add2be353aa46d5ab7e6c02b26f51f7'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:38.946692 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-88729a1c-515d-49bd-9464-c6afc5f039e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2add2be353aa46d5ab7e6c02b26f51f7'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:38.982940 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-88729a1c-515d-49bd-9464-c6afc5f039e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2add2be353aa46d5ab7e6c02b26f51f7'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:38.986989 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-88729a1c-515d-49bd-9464-c6afc5f039e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2add2be353aa46d5ab7e6c02b26f51f7'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:39.010962 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 865/1747] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:23:38 2022] DELETE /identity/v3/users/2add2be353aa46d5ab7e6c02b26f51f7 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:39.016058 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d22a1-d259-4ff2-981b-360b053fe93f None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:39.016570 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d22a1-d259-4ff2-981b-360b053fe93f None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:39.016794 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d22a1-d259-4ff2-981b-360b053fe93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:39.036090 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b6274-3a49-42fb-89f6-dba65019868f None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:39.036383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b6274-3a49-42fb-89f6-dba65019868f None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:39.036669 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b6274-3a49-42fb-89f6-dba65019868f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:39.063227 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-410d22a1-d259-4ff2-981b-360b053fe93f None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:39.071575 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-410d22a1-d259-4ff2-981b-360b053fe93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:39.087839 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:39.091718 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-934b6274-3a49-42fb-89f6-dba65019868f None None] MFA Rules not processed for user `a094273e734342eca2bdf159fab8a8d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:39.094584 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-934b6274-3a49-42fb-89f6-dba65019868f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:39.174630 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:39.323010 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 866/1748] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:39.335042 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 883/1749] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:39.346847 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934b6274-3a49-42fb-89f6-dba65019868f None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:39.349906 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8436e58-03a6-413f-9a2b-dbba78c8dde2 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:39.350198 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8436e58-03a6-413f-9a2b-dbba78c8dde2 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:39.350866 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-934b6274-3a49-42fb-89f6-dba65019868f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:39.352305 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8436e58-03a6-413f-9a2b-dbba78c8dde2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:39.453210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-a8436e58-03a6-413f-9a2b-dbba78c8dde2 None None] MFA Rules not processed for user `053412d72e6e4b4e8865d50a1594a3b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:39.454822 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-a8436e58-03a6-413f-9a2b-dbba78c8dde2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:39.653816 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934b6274-3a49-42fb-89f6-dba65019868f None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:39.674820 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4febcfb-a5c6-4760-ac5e-2ff210c95dcf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:39.675509 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4febcfb-a5c6-4760-ac5e-2ff210c95dcf service nova] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:39.675789 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4febcfb-a5c6-4760-ac5e-2ff210c95dcf service nova] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:39.676060 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4febcfb-a5c6-4760-ac5e-2ff210c95dcf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:39.709324 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4febcfb-a5c6-4760-ac5e-2ff210c95dcf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:39.715512 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4febcfb-a5c6-4760-ac5e-2ff210c95dcf service nova] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:39.722800 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 884/1750] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:39 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:39.729243 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8436e58-03a6-413f-9a2b-dbba78c8dde2 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:39.730951 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-a8436e58-03a6-413f-9a2b-dbba78c8dde2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:39.781971 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 867/1751] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:39.885965 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8436e58-03a6-413f-9a2b-dbba78c8dde2 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:39.898174 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee2c67-c0e3-43da-96c3-1fa94644af19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:39.903049 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee2c67-c0e3-43da-96c3-1fa94644af19 service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:39.903515 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee2c67-c0e3-43da-96c3-1fa94644af19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:39.904611 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee2c67-c0e3-43da-96c3-1fa94644af19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:39.944081 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ee2c67-c0e3-43da-96c3-1fa94644af19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:39.949930 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ee2c67-c0e3-43da-96c3-1fa94644af19 service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:40.001326 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 885/1752] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:41.522616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4febcfb-a5c6-4760-ac5e-2ff210c95dcf service nova] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:41.524743 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-b4febcfb-a5c6-4760-ac5e-2ff210c95dcf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:41.546824 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4febcfb-a5c6-4760-ac5e-2ff210c95dcf service nova] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:41.565772 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0618b8-2190-4dc6-a169-0ba022ba1e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:41.566938 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0618b8-2190-4dc6-a169-0ba022ba1e29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:41.567391 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0618b8-2190-4dc6-a169-0ba022ba1e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:41.567804 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0618b8-2190-4dc6-a169-0ba022ba1e29 admin admin] PATH_INFO: `/v3/users/3be0341c558f4e40b1e5b909c79a52ca` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:41.569276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0618b8-2190-4dc6-a169-0ba022ba1e29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3be0341c558f4e40b1e5b909c79a52ca)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:41.571449 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0618b8-2190-4dc6-a169-0ba022ba1e29 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:41.668215 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-df0618b8-2190-4dc6-a169-0ba022ba1e29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3be0341c558f4e40b1e5b909c79a52ca'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:41.686377 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-df0618b8-2190-4dc6-a169-0ba022ba1e29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3be0341c558f4e40b1e5b909c79a52ca'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:41.700980 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-df0618b8-2190-4dc6-a169-0ba022ba1e29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3be0341c558f4e40b1e5b909c79a52ca'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:41.716053 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-df0618b8-2190-4dc6-a169-0ba022ba1e29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3be0341c558f4e40b1e5b909c79a52ca'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:41.717108 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-df0618b8-2190-4dc6-a169-0ba022ba1e29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3be0341c558f4e40b1e5b909c79a52ca'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:41.725636 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 868/1753] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:23:41 2022] DELETE /identity/v3/users/3be0341c558f4e40b1e5b909c79a52ca => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:41.731108 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5584c5-c5d8-4bf0-943d-9705052804cc None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:41.731598 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5584c5-c5d8-4bf0-943d-9705052804cc None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:41.731690 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5584c5-c5d8-4bf0-943d-9705052804cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:41.756516 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-3d5584c5-c5d8-4bf0-943d-9705052804cc None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:41.757795 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-3d5584c5-c5d8-4bf0-943d-9705052804cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:41.823546 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:41.867330 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 886/1754] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:41.917192 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0618b8-2190-4dc6-a169-0ba022ba1e29 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:41.921917 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-df0618b8-2190-4dc6-a169-0ba022ba1e29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:41.978908 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:42.017209 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0618b8-2190-4dc6-a169-0ba022ba1e29 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:42.022258 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f7b87c-5751-4313-ad42-e35564a398fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:42.023619 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7b87c-5751-4313-ad42-e35564a398fe service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:42.025736 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7b87c-5751-4313-ad42-e35564a398fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:42.026316 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7b87c-5751-4313-ad42-e35564a398fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:42.038906 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f7b87c-5751-4313-ad42-e35564a398fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:42.041958 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f7b87c-5751-4313-ad42-e35564a398fe service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:42.055326 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 869/1755] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:42.309878 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d1692-7e69-4c28-aca7-f6556f2c2c7d None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:42.310637 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d1692-7e69-4c28-aca7-f6556f2c2c7d None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:42.310637 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d1692-7e69-4c28-aca7-f6556f2c2c7d None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:42.312325 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 887/1756] 10.222.0.50 () {54 vars in 921 bytes} [Fri May 27 03:23:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:23:42.381814 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9a6c1-6926-4be2-ab96-6bd71eb8113a None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:42.384320 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9a6c1-6926-4be2-ab96-6bd71eb8113a None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:42.385603 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9a6c1-6926-4be2-ab96-6bd71eb8113a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:42.437538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-8bd9a6c1-6926-4be2-ab96-6bd71eb8113a None None] MFA Rules not processed for user `30963f2f464e4f54a8aea0175370a235`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:42.449527 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-8bd9a6c1-6926-4be2-ab96-6bd71eb8113a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:42.554288 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5d1692-7e69-4c28-aca7-f6556f2c2c7d None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:42.556977 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-dd5d1692-7e69-4c28-aca7-f6556f2c2c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:42.594709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5d1692-7e69-4c28-aca7-f6556f2c2c7d None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:42.601913 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 870/1757] 10.222.0.50 () {58 vars in 1018 bytes} [Fri May 27 03:23:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:42.612756 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb5cc0f-58b1-4672-9299-b4ebbf53e142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:42.614117 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb5cc0f-58b1-4672-9299-b4ebbf53e142 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:42.614553 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb5cc0f-58b1-4672-9299-b4ebbf53e142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:42.615034 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb5cc0f-58b1-4672-9299-b4ebbf53e142 admin admin] PATH_INFO: `/v3/projects/f7a93f7132614f2093ab91d2b205f4e9` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:42.616434 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb5cc0f-58b1-4672-9299-b4ebbf53e142 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7a93f7132614f2093ab91d2b205f4e9)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:42.618953 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: WARNING py.warnings [None req-edb5cc0f-58b1-4672-9299-b4ebbf53e142 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 03:23:42.618953 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: warnings.warn(msg) May 27 03:23:42.618953 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: May 27 03:23:42.621389 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb5cc0f-58b1-4672-9299-b4ebbf53e142 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:42.691239 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9a6c1-6926-4be2-ab96-6bd71eb8113a None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:42.693194 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-8bd9a6c1-6926-4be2-ab96-6bd71eb8113a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:42.720301 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9a6c1-6926-4be2-ab96-6bd71eb8113a None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:42.766041 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-edb5cc0f-58b1-4672-9299-b4ebbf53e142 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7a93f7132614f2093ab91d2b205f4e9', 'request_id': 'req-edb5cc0f-58b1-4672-9299-b4ebbf53e142'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:42.809674 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-edb5cc0f-58b1-4672-9299-b4ebbf53e142 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7a93f7132614f2093ab91d2b205f4e9', 'request_id': 'req-edb5cc0f-58b1-4672-9299-b4ebbf53e142'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:42.838652 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-edb5cc0f-58b1-4672-9299-b4ebbf53e142 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7a93f7132614f2093ab91d2b205f4e9', 'request_id': 'req-edb5cc0f-58b1-4672-9299-b4ebbf53e142'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:42.857845 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-edb5cc0f-58b1-4672-9299-b4ebbf53e142 admin admin] The token cache is being invalidate because project f7a93f7132614f2093ab91d2b205f4e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:23:42.858133 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-edb5cc0f-58b1-4672-9299-b4ebbf53e142 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:42.860982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-edb5cc0f-58b1-4672-9299-b4ebbf53e142 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:42.866538 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 888/1758] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:23:42 2022] DELETE /identity/v3/projects/f7a93f7132614f2093ab91d2b205f4e9 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:42.872262 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b4e11-df80-4110-a910-47a51b3e15e6 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:42.872756 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b4e11-df80-4110-a910-47a51b3e15e6 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:42.873891 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9b4e11-df80-4110-a910-47a51b3e15e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:42.927708 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-1b9b4e11-df80-4110-a910-47a51b3e15e6 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:42.930886 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-1b9b4e11-df80-4110-a910-47a51b3e15e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:42.944933 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1767c-528c-4005-a1c9-8cef7978529c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30963f2f464e4f54a8aea0175370a235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:42.945773 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1767c-528c-4005-a1c9-8cef7978529c service placement] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:42.946078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1767c-528c-4005-a1c9-8cef7978529c service placement] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:42.946365 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1767c-528c-4005-a1c9-8cef7978529c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:42.951131 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-fbf1767c-528c-4005-a1c9-8cef7978529c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:42.983423 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1767c-528c-4005-a1c9-8cef7978529c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:42.985190 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf1767c-528c-4005-a1c9-8cef7978529c service placement] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:43.036398 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 871/1759] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:23:43.109468 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 889/1760] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:43.110017 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8ebae-c764-4a0f-a61b-74bd3f963f36 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:43.110309 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8ebae-c764-4a0f-a61b-74bd3f963f36 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:43.110587 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8ebae-c764-4a0f-a61b-74bd3f963f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:43.169783 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-acf8ebae-c764-4a0f-a61b-74bd3f963f36 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:43.171780 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-acf8ebae-c764-4a0f-a61b-74bd3f963f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:43.189063 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b4e11-df80-4110-a910-47a51b3e15e6 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:43.190583 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 872/1761] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:23:43.229036 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9b4e11-df80-4110-a910-47a51b3e15e6 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:43.242538 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f93d7-5fc6-413a-a2d7-699c1ac3f2a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:43.244168 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f93d7-5fc6-413a-a2d7-699c1ac3f2a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:43.244679 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f93d7-5fc6-413a-a2d7-699c1ac3f2a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:43.245207 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f93d7-5fc6-413a-a2d7-699c1ac3f2a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:43.246942 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509f93d7-5fc6-413a-a2d7-699c1ac3f2a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:23:43.247480 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509f93d7-5fc6-413a-a2d7-699c1ac3f2a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:43.250829 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509f93d7-5fc6-413a-a2d7-699c1ac3f2a0 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:43.320984 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf8ebae-c764-4a0f-a61b-74bd3f963f36 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:43.323151 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-acf8ebae-c764-4a0f-a61b-74bd3f963f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:43.345608 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 890/1762] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:23:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:23:43.346806 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f93d7-5fc6-413a-a2d7-699c1ac3f2a0 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:43.348398 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf8ebae-c764-4a0f-a61b-74bd3f963f36 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:43.360842 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f93d7-5fc6-413a-a2d7-699c1ac3f2a0 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:43.362944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615c2f4-7bf9-450e-8f99-6f0c7f6f04ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:43.363884 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615c2f4-7bf9-450e-8f99-6f0c7f6f04ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:43.364168 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615c2f4-7bf9-450e-8f99-6f0c7f6f04ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:43.364503 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615c2f4-7bf9-450e-8f99-6f0c7f6f04ea admin admin] PATH_INFO: `/v3/projects/bce58ee1ad2f4ac4a0afbfd819d2cea0` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:43.366078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6615c2f4-7bf9-450e-8f99-6f0c7f6f04ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=bce58ee1ad2f4ac4a0afbfd819d2cea0)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:43.367459 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98dfdec-ecbc-4116-8c96-2115069a14d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:43.368399 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98dfdec-ecbc-4116-8c96-2115069a14d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:43.368886 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98dfdec-ecbc-4116-8c96-2115069a14d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:43.370179 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98dfdec-ecbc-4116-8c96-2115069a14d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:43.371215 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98dfdec-ecbc-4116-8c96-2115069a14d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:43.371322 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6615c2f4-7bf9-450e-8f99-6f0c7f6f04ea admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:43.374188 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98dfdec-ecbc-4116-8c96-2115069a14d4 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:43.484103 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 891/1763] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:23:43 2022] POST /identity/v3/projects => generated 381 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:43.487906 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98dfdec-ecbc-4116-8c96-2115069a14d4 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:43.519442 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-6615c2f4-7bf9-450e-8f99-6f0c7f6f04ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bce58ee1ad2f4ac4a0afbfd819d2cea0', 'request_id': 'req-6615c2f4-7bf9-450e-8f99-6f0c7f6f04ea'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:43.521337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:43.552583 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-6615c2f4-7bf9-450e-8f99-6f0c7f6f04ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bce58ee1ad2f4ac4a0afbfd819d2cea0', 'request_id': 'req-6615c2f4-7bf9-450e-8f99-6f0c7f6f04ea'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:43.566544 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-6615c2f4-7bf9-450e-8f99-6f0c7f6f04ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bce58ee1ad2f4ac4a0afbfd819d2cea0', 'request_id': 'req-6615c2f4-7bf9-450e-8f99-6f0c7f6f04ea'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:43.595939 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-6615c2f4-7bf9-450e-8f99-6f0c7f6f04ea admin admin] The token cache is being invalidate because project bce58ee1ad2f4ac4a0afbfd819d2cea0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:23:43.596344 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-6615c2f4-7bf9-450e-8f99-6f0c7f6f04ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:43.597331 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-6615c2f4-7bf9-450e-8f99-6f0c7f6f04ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:43.602362 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:43.603725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 873/1764] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:23:43 2022] DELETE /identity/v3/projects/bce58ee1ad2f4ac4a0afbfd819d2cea0 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:23:43.605306 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10dffd-1d52-427d-97a8-cb10a562bb05 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:43.605688 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10dffd-1d52-427d-97a8-cb10a562bb05 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:43.609519 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10dffd-1d52-427d-97a8-cb10a562bb05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:43.658626 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-1b10dffd-1d52-427d-97a8-cb10a562bb05 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:43.662044 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-1b10dffd-1d52-427d-97a8-cb10a562bb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:43.721112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98dfdec-ecbc-4116-8c96-2115069a14d4 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:43.831843 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 874/1765] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:43.842346 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a1344f-bd14-4fb6-9bff-712596a29cbc None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:43.845455 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a1344f-bd14-4fb6-9bff-712596a29cbc None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:43.845888 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a1344f-bd14-4fb6-9bff-712596a29cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:43.871135 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303f2a6b-109e-41cd-988b-f2292457d4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:43.872488 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f2a6b-109e-41cd-988b-f2292457d4e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:43.872779 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f2a6b-109e-41cd-988b-f2292457d4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:43.873088 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303f2a6b-109e-41cd-988b-f2292457d4e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:43.873812 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303f2a6b-109e-41cd-988b-f2292457d4e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:43.878362 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303f2a6b-109e-41cd-988b-f2292457d4e0 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:43.885319 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-09a1344f-bd14-4fb6-9bff-712596a29cbc None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:43.886576 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-09a1344f-bd14-4fb6-9bff-712596a29cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:43.916868 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 875/1766] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:43.920936 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a1344f-bd14-4fb6-9bff-712596a29cbc None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:43.924146 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 892/1767] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:23:43 2022] POST /identity/v3/users => generated 291 bytes in 439 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:43.928175 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303f2a6b-109e-41cd-988b-f2292457d4e0 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:43.932174 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-303f2a6b-109e-41cd-988b-f2292457d4e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:43.940459 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a1344f-bd14-4fb6-9bff-712596a29cbc None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:43.954048 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df5000-dbd1-4670-ba1e-6cada6fa73fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:43.954880 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df5000-dbd1-4670-ba1e-6cada6fa73fd admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:43.955175 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df5000-dbd1-4670-ba1e-6cada6fa73fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:43.955457 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df5000-dbd1-4670-ba1e-6cada6fa73fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:43.956265 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df5000-dbd1-4670-ba1e-6cada6fa73fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:23:43.956572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df5000-dbd1-4670-ba1e-6cada6fa73fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:43.957386 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303f2a6b-109e-41cd-988b-f2292457d4e0 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:43.958424 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df5000-dbd1-4670-ba1e-6cada6fa73fd admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:43.971168 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c127ae-78cf-49ae-953c-19982ed9d7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:43.973487 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c127ae-78cf-49ae-953c-19982ed9d7b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:43.975025 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c127ae-78cf-49ae-953c-19982ed9d7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:43.975316 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c127ae-78cf-49ae-953c-19982ed9d7b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:43.978492 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c127ae-78cf-49ae-953c-19982ed9d7b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:43.980222 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c127ae-78cf-49ae-953c-19982ed9d7b4 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:43.983146 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 876/1768] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:23:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:23:43.987132 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df5000-dbd1-4670-ba1e-6cada6fa73fd admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:44.010583 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df5000-dbd1-4670-ba1e-6cada6fa73fd admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:44.022615 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 893/1769] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:44.028087 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df047168-62a4-4e4e-b8fe-dbfbb39d31d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:44.028786 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df047168-62a4-4e4e-b8fe-dbfbb39d31d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:44.029095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df047168-62a4-4e4e-b8fe-dbfbb39d31d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:44.029368 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df047168-62a4-4e4e-b8fe-dbfbb39d31d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:44.030081 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df047168-62a4-4e4e-b8fe-dbfbb39d31d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:44.032038 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df047168-62a4-4e4e-b8fe-dbfbb39d31d3 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:44.044591 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c127ae-78cf-49ae-953c-19982ed9d7b4 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:44.086746 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c127ae-78cf-49ae-953c-19982ed9d7b4 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:44.105992 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 877/1770] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:23:43 2022] POST /identity/v3/projects => generated 377 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:23:44.107065 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df047168-62a4-4e4e-b8fe-dbfbb39d31d3 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:44.107204 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39efe52f-b889-4798-89de-23e7acabecfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:44.107909 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efe52f-b889-4798-89de-23e7acabecfc admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:44.108128 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efe52f-b889-4798-89de-23e7acabecfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:44.108440 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efe52f-b889-4798-89de-23e7acabecfc admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:44.109481 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39efe52f-b889-4798-89de-23e7acabecfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:44.111121 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39efe52f-b889-4798-89de-23e7acabecfc admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:44.138666 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 894/1771] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:44.140292 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b2e45-3b56-47e4-bcfb-5688ae9b844e None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:44.140921 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b2e45-3b56-47e4-bcfb-5688ae9b844e None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:44.141621 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b2e45-3b56-47e4-bcfb-5688ae9b844e None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:44.149565 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 895/1772] 10.222.0.50 () {54 vars in 921 bytes} [Fri May 27 03:23:44 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:23:44.150571 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b2e45-3b56-47e4-bcfb-5688ae9b844e None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:44.248332 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df047168-62a4-4e4e-b8fe-dbfbb39d31d3 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:44.268492 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcf0ac2-1001-4f33-b9b7-6f8b0c967244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:44.269161 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf0ac2-1001-4f33-b9b7-6f8b0c967244 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:44.269446 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf0ac2-1001-4f33-b9b7-6f8b0c967244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:44.269727 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf0ac2-1001-4f33-b9b7-6f8b0c967244 admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:44.270439 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcf0ac2-1001-4f33-b9b7-6f8b0c967244 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:44.272427 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcf0ac2-1001-4f33-b9b7-6f8b0c967244 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:44.323111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b2e45-3b56-47e4-bcfb-5688ae9b844e None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:44.333022 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 878/1773] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:23:44 2022] POST /identity/v3/users => generated 289 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:44.333022 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f024e18b-24fb-4166-9e25-2868b3713724 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:44.333022 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f024e18b-24fb-4166-9e25-2868b3713724 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:44.333022 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f024e18b-24fb-4166-9e25-2868b3713724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:44.343494 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c67b3-08a1-4f2a-b676-86de30eb9763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:44.346150 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c67b3-08a1-4f2a-b676-86de30eb9763 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:44.346526 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c67b3-08a1-4f2a-b676-86de30eb9763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:44.346946 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8c67b3-08a1-4f2a-b676-86de30eb9763 admin admin] PATH_INFO: `/v3/projects/8f95e0c579a7421cbe237f5715de337d/users/91978f5026db4454adba2bf18ecd917b/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:44.382584 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c67b3-08a1-4f2a-b676-86de30eb9763 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f95e0c579a7421cbe237f5715de337d, user_id=91978f5026db4454adba2bf18ecd917b, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:44.386426 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8c67b3-08a1-4f2a-b676-86de30eb9763 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:44.390041 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-f024e18b-24fb-4166-9e25-2868b3713724 None None] MFA Rules not processed for user `1a699988f08a4001b3b7d764f01f4fc5`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:44.391644 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-f024e18b-24fb-4166-9e25-2868b3713724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:44.417324 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 896/1774] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:23:44 2022] PUT /identity/v3/projects/8f95e0c579a7421cbe237f5715de337d/users/91978f5026db4454adba2bf18ecd917b/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:44.418740 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c67b3-08a1-4f2a-b676-86de30eb9763 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:44.441206 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:44.480942 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:44.548706 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8c67b3-08a1-4f2a-b676-86de30eb9763 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:44.553192 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 879/1775] 10.222.0.50 () {58 vars in 1018 bytes} [Fri May 27 03:23:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:44.554761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef0c03-2306-4b36-b87b-e4f5757221e6 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:44.554941 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eab29c-9127-4802-9e90-df77603ff014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:44.556510 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eab29c-9127-4802-9e90-df77603ff014 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:44.556968 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef0c03-2306-4b36-b87b-e4f5757221e6 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:44.557209 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eab29c-9127-4802-9e90-df77603ff014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:44.557449 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef0c03-2306-4b36-b87b-e4f5757221e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:44.557557 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eab29c-9127-4802-9e90-df77603ff014 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:44.558274 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eab29c-9127-4802-9e90-df77603ff014 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:44.561240 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78eab29c-9127-4802-9e90-df77603ff014 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:44.583889 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 897/1776] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:44.585910 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eab29c-9127-4802-9e90-df77603ff014 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:44.603593 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eab29c-9127-4802-9e90-df77603ff014 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:44.612564 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8236336a-df0f-49a7-8d85-1faee92535eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:44.613340 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8236336a-df0f-49a7-8d85-1faee92535eb admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:44.613621 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8236336a-df0f-49a7-8d85-1faee92535eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:44.613904 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8236336a-df0f-49a7-8d85-1faee92535eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:44.614855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8236336a-df0f-49a7-8d85-1faee92535eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:44.618394 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8236336a-df0f-49a7-8d85-1faee92535eb admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:44.631583 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-45ef0c03-2306-4b36-b87b-e4f5757221e6 None None] MFA Rules not processed for user `91978f5026db4454adba2bf18ecd917b`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:44.632925 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-45ef0c03-2306-4b36-b87b-e4f5757221e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:44.638472 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 898/1777] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:44.639923 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8236336a-df0f-49a7-8d85-1faee92535eb admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:44.642421 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-8236336a-df0f-49a7-8d85-1faee92535eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:44.767913 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8236336a-df0f-49a7-8d85-1faee92535eb admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:44.779612 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258c258-a7bd-4574-88c8-c1d168af6a1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:44.780557 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258c258-a7bd-4574-88c8-c1d168af6a1c service nova] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:44.781032 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258c258-a7bd-4574-88c8-c1d168af6a1c service nova] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:44.781479 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258c258-a7bd-4574-88c8-c1d168af6a1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:44.805935 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e258c258-a7bd-4574-88c8-c1d168af6a1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:44.808385 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e258c258-a7bd-4574-88c8-c1d168af6a1c service nova] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:44.840911 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 899/1778] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:44.842232 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258c258-a7bd-4574-88c8-c1d168af6a1c service nova] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:44.865483 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258c258-a7bd-4574-88c8-c1d168af6a1c service nova] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:44.871627 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa4ea1-0d99-4765-8e1e-5edcbe266282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:44.873479 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 880/1779] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:44.873805 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa4ea1-0d99-4765-8e1e-5edcbe266282 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:44.874231 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa4ea1-0d99-4765-8e1e-5edcbe266282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:44.874649 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa4ea1-0d99-4765-8e1e-5edcbe266282 admin admin] PATH_INFO: `/v3/projects/1c488f6c9420498281252528d879a29b/users/235dd34113d043289d44e51419838b11/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:44.878386 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef0c03-2306-4b36-b87b-e4f5757221e6 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:44.906938 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef0c03-2306-4b36-b87b-e4f5757221e6 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:44.909595 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aa4ea1-0d99-4765-8e1e-5edcbe266282 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c488f6c9420498281252528d879a29b, user_id=235dd34113d043289d44e51419838b11, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:44.913039 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aa4ea1-0d99-4765-8e1e-5edcbe266282 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:44.917631 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da750bde-2ea4-439f-b107-d579fb6c368b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:44.918433 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da750bde-2ea4-439f-b107-d579fb6c368b admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:44.918814 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da750bde-2ea4-439f-b107-d579fb6c368b admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:44.919181 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da750bde-2ea4-439f-b107-d579fb6c368b admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:44.920108 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da750bde-2ea4-439f-b107-d579fb6c368b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:44.926747 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da750bde-2ea4-439f-b107-d579fb6c368b admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:44.942084 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 900/1780] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:23:44 2022] PUT /identity/v3/projects/1c488f6c9420498281252528d879a29b/users/235dd34113d043289d44e51419838b11/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:44.952712 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f3707-e717-465a-be44-705dd4c34920 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:44.953410 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f3707-e717-465a-be44-705dd4c34920 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:44.953816 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f3707-e717-465a-be44-705dd4c34920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:44.998424 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 881/1781] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:23:44 2022] POST /identity/v3/projects => generated 381 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:45.004992 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da750bde-2ea4-439f-b107-d579fb6c368b admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:45.010576 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-f17f3707-e717-465a-be44-705dd4c34920 None None] MFA Rules not processed for user `235dd34113d043289d44e51419838b11`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:45.017710 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-f17f3707-e717-465a-be44-705dd4c34920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:45.133875 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da750bde-2ea4-439f-b107-d579fb6c368b admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:45.138303 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 901/1782] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:23:45.139867 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f3707-e717-465a-be44-705dd4c34920 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:45.145895 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e50e5e5-abb2-45d2-9e14-8fce5abcd6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:45.146812 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50e5e5-abb2-45d2-9e14-8fce5abcd6d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:45.147240 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50e5e5-abb2-45d2-9e14-8fce5abcd6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:45.148044 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:45.148620 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50e5e5-abb2-45d2-9e14-8fce5abcd6d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:45.149949 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e50e5e5-abb2-45d2-9e14-8fce5abcd6d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:45.157760 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f3707-e717-465a-be44-705dd4c34920 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:45.158247 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e50e5e5-abb2-45d2-9e14-8fce5abcd6d1 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:45.166359 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8cd990-efd7-49ba-8ce6-a4979084acf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:45.167208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8cd990-efd7-49ba-8ce6-a4979084acf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:45.168445 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8cd990-efd7-49ba-8ce6-a4979084acf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:45.169104 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8cd990-efd7-49ba-8ce6-a4979084acf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:45.170551 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8cd990-efd7-49ba-8ce6-a4979084acf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:45.172170 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8cd990-efd7-49ba-8ce6-a4979084acf3 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:45.209053 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 882/1783] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:23:45 2022] POST /identity/v3/users => generated 297 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:45.216437 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e50e5e5-abb2-45d2-9e14-8fce5abcd6d1 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:45.224725 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 902/1784] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:23:45 2022] POST /identity/v3/projects => generated 375 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:45.229940 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8cd990-efd7-49ba-8ce6-a4979084acf3 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:45.236080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e50e5e5-abb2-45d2-9e14-8fce5abcd6d1 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:45.275275 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:45.348831 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8cd990-efd7-49ba-8ce6-a4979084acf3 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:45.371185 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc992a-432e-4087-9a1b-87129fd9de0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:45.371467 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6472ea99-387c-4643-8a7b-a72f01ec6e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:45.372227 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc992a-432e-4087-9a1b-87129fd9de0b admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:45.372667 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc992a-432e-4087-9a1b-87129fd9de0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:45.373010 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472ea99-387c-4643-8a7b-a72f01ec6e6f admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:45.373202 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc992a-432e-4087-9a1b-87129fd9de0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:45.374335 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472ea99-387c-4643-8a7b-a72f01ec6e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:45.374755 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472ea99-387c-4643-8a7b-a72f01ec6e6f admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:45.375399 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edc992a-432e-4087-9a1b-87129fd9de0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:45.376464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6472ea99-387c-4643-8a7b-a72f01ec6e6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:45.377875 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edc992a-432e-4087-9a1b-87129fd9de0b admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:45.382391 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6472ea99-387c-4643-8a7b-a72f01ec6e6f admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:45.402049 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 883/1785] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:45.406249 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc992a-432e-4087-9a1b-87129fd9de0b admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:45.424102 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc992a-432e-4087-9a1b-87129fd9de0b admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:45.443274 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 903/1786] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:23:45 2022] POST /identity/v3/users => generated 288 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:45.443501 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec46f3-789c-4702-a790-96c781ece8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:45.443501 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec46f3-789c-4702-a790-96c781ece8ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:45.443501 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec46f3-789c-4702-a790-96c781ece8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:45.443501 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec46f3-789c-4702-a790-96c781ece8ed admin admin] PATH_INFO: `/v3/projects/9115fc4f8d5b4718a74c1e1497e26f41/users/e42cd4d5bc064da0bbb76e3b1f869c56/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:45.444826 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6472ea99-387c-4643-8a7b-a72f01ec6e6f admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:45.472111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6472ea99-387c-4643-8a7b-a72f01ec6e6f admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:45.477573 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c04bec8-f211-41ba-b190-6ea8b83a700d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:45.477799 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ec46f3-789c-4702-a790-96c781ece8ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=9115fc4f8d5b4718a74c1e1497e26f41, user_id=e42cd4d5bc064da0bbb76e3b1f869c56, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:45.478477 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04bec8-f211-41ba-b190-6ea8b83a700d admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:45.478757 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04bec8-f211-41ba-b190-6ea8b83a700d admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:45.479028 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c04bec8-f211-41ba-b190-6ea8b83a700d admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:45.479756 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c04bec8-f211-41ba-b190-6ea8b83a700d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:45.480646 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ec46f3-789c-4702-a790-96c781ece8ed admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:45.481972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c04bec8-f211-41ba-b190-6ea8b83a700d admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:45.497394 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 904/1787] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:45.505964 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c04bec8-f211-41ba-b190-6ea8b83a700d admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:45.514772 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 884/1788] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:23:45 2022] PUT /identity/v3/projects/9115fc4f8d5b4718a74c1e1497e26f41/users/e42cd4d5bc064da0bbb76e3b1f869c56/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:45.519051 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec46f3-789c-4702-a790-96c781ece8ed admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:45.541859 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c04bec8-f211-41ba-b190-6ea8b83a700d admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:45.680989 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec46f3-789c-4702-a790-96c781ece8ed admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:45.687250 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c8777c-2026-46f7-9c56-f9622b1e420c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:45.688163 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8777c-2026-46f7-9c56-f9622b1e420c admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:45.688695 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8777c-2026-46f7-9c56-f9622b1e420c admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:45.689261 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c8777c-2026-46f7-9c56-f9622b1e420c admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:45.690618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c8777c-2026-46f7-9c56-f9622b1e420c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:45.692486 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c8777c-2026-46f7-9c56-f9622b1e420c admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:45.700301 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2823093-774b-4a8b-ac39-d497b92cd4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:45.701028 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2823093-774b-4a8b-ac39-d497b92cd4c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:45.701344 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2823093-774b-4a8b-ac39-d497b92cd4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:45.701669 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2823093-774b-4a8b-ac39-d497b92cd4c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:45.702722 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2823093-774b-4a8b-ac39-d497b92cd4c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:45.705798 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2823093-774b-4a8b-ac39-d497b92cd4c3 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:45.713910 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 885/1789] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:45.719149 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2c5af-954d-4d16-863e-a4831f01fe1e None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:45.719149 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2c5af-954d-4d16-863e-a4831f01fe1e None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:45.719516 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2c5af-954d-4d16-863e-a4831f01fe1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:45.724373 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 905/1790] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:45.725655 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2823093-774b-4a8b-ac39-d497b92cd4c3 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:45.742179 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2823093-774b-4a8b-ac39-d497b92cd4c3 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:45.749061 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de9f33-e1cb-41bd-bf9d-cf2e7853d01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:45.750057 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de9f33-e1cb-41bd-bf9d-cf2e7853d01f admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:45.750376 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de9f33-e1cb-41bd-bf9d-cf2e7853d01f admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:45.750660 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de9f33-e1cb-41bd-bf9d-cf2e7853d01f admin admin] PATH_INFO: `/v3/projects/9115fc4f8d5b4718a74c1e1497e26f41/users/e42cd4d5bc064da0bbb76e3b1f869c56/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:45.753320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66de9f33-e1cb-41bd-bf9d-cf2e7853d01f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9115fc4f8d5b4718a74c1e1497e26f41, user_id=e42cd4d5bc064da0bbb76e3b1f869c56, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:45.757567 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66de9f33-e1cb-41bd-bf9d-cf2e7853d01f admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:45.758512 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-0aa2c5af-954d-4d16-863e-a4831f01fe1e None None] MFA Rules not processed for user `19457c759f0b4f5cbc530364af1f47d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:45.760304 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-0aa2c5af-954d-4d16-863e-a4831f01fe1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:45.779351 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 906/1791] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:23:45 2022] PUT /identity/v3/projects/9115fc4f8d5b4718a74c1e1497e26f41/users/e42cd4d5bc064da0bbb76e3b1f869c56/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:45.780308 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de9f33-e1cb-41bd-bf9d-cf2e7853d01f admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:45.888226 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:45.890284 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 886/1792] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:23:45.894300 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9afff-46ab-41df-9e30-2e5986fddf5b None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:45.894855 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9afff-46ab-41df-9e30-2e5986fddf5b None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:45.895982 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f9afff-46ab-41df-9e30-2e5986fddf5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:45.943533 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de9f33-e1cb-41bd-bf9d-cf2e7853d01f admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:45.972821 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b38ca0-c53f-4685-9681-87a751ae9bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:45.978046 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b38ca0-c53f-4685-9681-87a751ae9bfd admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:45.980918 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b38ca0-c53f-4685-9681-87a751ae9bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:45.981548 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b38ca0-c53f-4685-9681-87a751ae9bfd admin admin] PATH_INFO: `/v3/projects/2f3a2da744c648ffa7f50045955574be/users/a6776c73da2f4b7a885af4fa99ba3d13/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:45.991853 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-38f9afff-46ab-41df-9e30-2e5986fddf5b None None] MFA Rules not processed for user `e42cd4d5bc064da0bbb76e3b1f869c56`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:45.993975 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-38f9afff-46ab-41df-9e30-2e5986fddf5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:46.020976 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:46.025547 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b38ca0-c53f-4685-9681-87a751ae9bfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f3a2da744c648ffa7f50045955574be, user_id=a6776c73da2f4b7a885af4fa99ba3d13, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:46.029892 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b38ca0-c53f-4685-9681-87a751ae9bfd admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:46.057406 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 907/1793] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:23:45 2022] PUT /identity/v3/projects/2f3a2da744c648ffa7f50045955574be/users/a6776c73da2f4b7a885af4fa99ba3d13/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:46.067961 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a1db1-00ae-420f-93a4-9b7980a5e84c None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:46.068246 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a1db1-00ae-420f-93a4-9b7980a5e84c None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:46.068534 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38a1db1-00ae-420f-93a4-9b7980a5e84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:46.135216 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-c38a1db1-00ae-420f-93a4-9b7980a5e84c None None] MFA Rules not processed for user `a6776c73da2f4b7a885af4fa99ba3d13`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:46.136472 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-c38a1db1-00ae-420f-93a4-9b7980a5e84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:46.148935 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 887/1794] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:45 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:46.160708 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f9afff-46ab-41df-9e30-2e5986fddf5b None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:46.162785 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-38f9afff-46ab-41df-9e30-2e5986fddf5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:46.282495 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 908/1795] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:23:46.285009 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369ddb7-bd7f-4444-8f25-479e34e6d54f None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:46.285301 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369ddb7-bd7f-4444-8f25-479e34e6d54f None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:46.287733 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369ddb7-bd7f-4444-8f25-479e34e6d54f None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:46.305083 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 909/1796] 10.222.0.50 () {54 vars in 921 bytes} [Fri May 27 03:23:46 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:23:46.306658 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7369ddb7-bd7f-4444-8f25-479e34e6d54f None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:46.333871 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f9afff-46ab-41df-9e30-2e5986fddf5b None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:46.485409 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7369ddb7-bd7f-4444-8f25-479e34e6d54f None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:46.491659 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a0dc86-65eb-46a4-af9e-2962f705b4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:46.492533 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0dc86-65eb-46a4-af9e-2962f705b4d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:46.492915 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0dc86-65eb-46a4-af9e-2962f705b4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:46.493644 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a0dc86-65eb-46a4-af9e-2962f705b4d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:46.494487 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a0dc86-65eb-46a4-af9e-2962f705b4d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:46.497281 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a0dc86-65eb-46a4-af9e-2962f705b4d1 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:46.507772 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587bee18-ffff-43dc-a676-0cd8c153b4db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:46.517916 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bee18-ffff-43dc-a676-0cd8c153b4db service cinder] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:46.518475 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bee18-ffff-43dc-a676-0cd8c153b4db service cinder] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:46.518777 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bee18-ffff-43dc-a676-0cd8c153b4db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:46.596210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 910/1797] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:23:46 2022] POST /identity/v3/projects => generated 377 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:46.598662 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e76df-d27d-4c07-8559-d81093496865 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:46.602890 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e76df-d27d-4c07-8559-d81093496865 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:46.603255 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e76df-d27d-4c07-8559-d81093496865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:46.673008 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-a48e76df-d27d-4c07-8559-d81093496865 None None] MFA Rules not processed for user `30963f2f464e4f54a8aea0175370a235`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:46.678086 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-a48e76df-d27d-4c07-8559-d81093496865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:46.747158 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587bee18-ffff-43dc-a676-0cd8c153b4db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:46.748650 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587bee18-ffff-43dc-a676-0cd8c153b4db service cinder] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:46.767374 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:46.784419 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 888/1798] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:46 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:46.787543 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587bee18-ffff-43dc-a676-0cd8c153b4db service cinder] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:46.793042 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-587bee18-ffff-43dc-a676-0cd8c153b4db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:46.819280 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587bee18-ffff-43dc-a676-0cd8c153b4db service cinder] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:46.876661 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 911/1799] 10.222.0.50 () {58 vars in 1018 bytes} [Fri May 27 03:23:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:46.877609 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e76df-d27d-4c07-8559-d81093496865 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:46.886591 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:46.989847 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e76df-d27d-4c07-8559-d81093496865 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:46.996374 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7829bdd-2a85-4e90-be15-2cd10baa80b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:46.997248 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7829bdd-2a85-4e90-be15-2cd10baa80b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:46.997767 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7829bdd-2a85-4e90-be15-2cd10baa80b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:46.998152 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7829bdd-2a85-4e90-be15-2cd10baa80b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:46.998937 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7829bdd-2a85-4e90-be15-2cd10baa80b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:47.002064 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7829bdd-2a85-4e90-be15-2cd10baa80b7 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:47.005970 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ea6a6-2980-48b1-83ee-8c0126a2cbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:47.006893 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ea6a6-2980-48b1-83ee-8c0126a2cbb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:47.007227 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ea6a6-2980-48b1-83ee-8c0126a2cbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:47.007488 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ea6a6-2980-48b1-83ee-8c0126a2cbb5 admin admin] PATH_INFO: `/v3/users/03af57e57529401ca3ce3b78ace5be3e` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:47.009406 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4ea6a6-2980-48b1-83ee-8c0126a2cbb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03af57e57529401ca3ce3b78ace5be3e)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:47.011371 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4ea6a6-2980-48b1-83ee-8c0126a2cbb5 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:47.037866 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 912/1800] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:23:46 2022] POST /identity/v3/users => generated 295 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:47.039281 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7829bdd-2a85-4e90-be15-2cd10baa80b7 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:47.129448 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-0d4ea6a6-2980-48b1-83ee-8c0126a2cbb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03af57e57529401ca3ce3b78ace5be3e'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:47.141832 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7829bdd-2a85-4e90-be15-2cd10baa80b7 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:47.146855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721fb1fe-b0a1-49f7-b330-206021409568 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:47.147561 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fb1fe-b0a1-49f7-b330-206021409568 service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:47.148558 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fb1fe-b0a1-49f7-b330-206021409568 service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:47.150107 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fb1fe-b0a1-49f7-b330-206021409568 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:47.161995 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-0d4ea6a6-2980-48b1-83ee-8c0126a2cbb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03af57e57529401ca3ce3b78ace5be3e'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:47.191427 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-0d4ea6a6-2980-48b1-83ee-8c0126a2cbb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03af57e57529401ca3ce3b78ace5be3e'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:47.208424 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-0d4ea6a6-2980-48b1-83ee-8c0126a2cbb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03af57e57529401ca3ce3b78ace5be3e'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:47.211088 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-0d4ea6a6-2980-48b1-83ee-8c0126a2cbb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03af57e57529401ca3ce3b78ace5be3e'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:47.225446 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 889/1801] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:23:46 2022] DELETE /identity/v3/users/03af57e57529401ca3ce3b78ace5be3e => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:47.226449 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ea6a6-2980-48b1-83ee-8c0126a2cbb5 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:47.231081 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-0d4ea6a6-2980-48b1-83ee-8c0126a2cbb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:47.245334 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721fb1fe-b0a1-49f7-b330-206021409568 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:47.247023 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721fb1fe-b0a1-49f7-b330-206021409568 service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:47.283183 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 913/1802] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:47 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:47.284305 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721fb1fe-b0a1-49f7-b330-206021409568 service cinder] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:47.304026 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721fb1fe-b0a1-49f7-b330-206021409568 service cinder] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:47.338739 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:47.398622 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ea6a6-2980-48b1-83ee-8c0126a2cbb5 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:47.414446 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c57a2e-e798-4a7e-b7fc-e1eea33d4f01 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30963f2f464e4f54a8aea0175370a235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:47.415194 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c57a2e-e798-4a7e-b7fc-e1eea33d4f01 service placement] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:47.415300 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c57a2e-e798-4a7e-b7fc-e1eea33d4f01 service placement] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:47.415611 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c57a2e-e798-4a7e-b7fc-e1eea33d4f01 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:47.417874 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d569ed-c4c1-46c9-8fc5-c6cc765c2eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:47.418030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-50c57a2e-e798-4a7e-b7fc-e1eea33d4f01 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:47.419260 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d569ed-c4c1-46c9-8fc5-c6cc765c2eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:47.419574 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d569ed-c4c1-46c9-8fc5-c6cc765c2eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:47.419995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d569ed-c4c1-46c9-8fc5-c6cc765c2eb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:47.421589 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d569ed-c4c1-46c9-8fc5-c6cc765c2eb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:47.424126 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d569ed-c4c1-46c9-8fc5-c6cc765c2eb2 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:47.440594 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 914/1803] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:47 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:47.442447 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94acec-8f02-4285-a68c-a2e7cda03a4f None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:47.443609 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94acec-8f02-4285-a68c-a2e7cda03a4f None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:47.444132 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d94acec-8f02-4285-a68c-a2e7cda03a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:47.455798 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c57a2e-e798-4a7e-b7fc-e1eea33d4f01 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:47.460217 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c57a2e-e798-4a7e-b7fc-e1eea33d4f01 service placement] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:47.465111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-5d94acec-8f02-4285-a68c-a2e7cda03a4f None None] MFA Rules not processed for user `dd9938d83dc84ae494310741e2cba2ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:47.466599 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-5d94acec-8f02-4285-a68c-a2e7cda03a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:47.488643 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 890/1804] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:47.489638 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c57a2e-e798-4a7e-b7fc-e1eea33d4f01 service placement] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:47.491097 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-50c57a2e-e798-4a7e-b7fc-e1eea33d4f01 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:47.524689 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c57a2e-e798-4a7e-b7fc-e1eea33d4f01 service placement] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:47.539330 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925a204-e987-4878-8361-a9d1240f4c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:47.540525 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925a204-e987-4878-8361-a9d1240f4c6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:47.540525 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925a204-e987-4878-8361-a9d1240f4c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:47.541022 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925a204-e987-4878-8361-a9d1240f4c6b admin admin] PATH_INFO: `/v3/projects/350c2f8f5adb4b44992dd12b32d5d11e/users/158489cfd87e4d2cb82d07e45b641aed/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:47.569789 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 915/1805] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:47.589620 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c925a204-e987-4878-8361-a9d1240f4c6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=350c2f8f5adb4b44992dd12b32d5d11e, user_id=158489cfd87e4d2cb82d07e45b641aed, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:47.594364 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c925a204-e987-4878-8361-a9d1240f4c6b admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:47.622495 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 891/1806] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:23:47 2022] PUT /identity/v3/projects/350c2f8f5adb4b44992dd12b32d5d11e/users/158489cfd87e4d2cb82d07e45b641aed/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:47.626261 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d94acec-8f02-4285-a68c-a2e7cda03a4f None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:47.639401 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d94acec-8f02-4285-a68c-a2e7cda03a4f None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:47.706058 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dbe4f0-e86c-49d0-a83a-91ba229d21f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:47.707230 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dbe4f0-e86c-49d0-a83a-91ba229d21f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:47.707703 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dbe4f0-e86c-49d0-a83a-91ba229d21f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:47.708176 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dbe4f0-e86c-49d0-a83a-91ba229d21f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:47.709240 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dbe4f0-e86c-49d0-a83a-91ba229d21f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:47.713341 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dbe4f0-e86c-49d0-a83a-91ba229d21f7 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:47.730784 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 916/1807] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:47 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:47.737077 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925a204-e987-4878-8361-a9d1240f4c6b admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:47.751938 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:47.756569 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925a204-e987-4878-8361-a9d1240f4c6b admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:47.769981 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85641ae-d18f-4ee6-b0c9-5a9b03d0e8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:47.771154 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85641ae-d18f-4ee6-b0c9-5a9b03d0e8cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:47.771611 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85641ae-d18f-4ee6-b0c9-5a9b03d0e8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:47.771952 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85641ae-d18f-4ee6-b0c9-5a9b03d0e8cd admin admin] PATH_INFO: `/v3/projects/350c2f8f5adb4b44992dd12b32d5d11e/users/158489cfd87e4d2cb82d07e45b641aed/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:47.774223 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85641ae-d18f-4ee6-b0c9-5a9b03d0e8cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=350c2f8f5adb4b44992dd12b32d5d11e, user_id=158489cfd87e4d2cb82d07e45b641aed, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:47.777521 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85641ae-d18f-4ee6-b0c9-5a9b03d0e8cd admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:47.801497 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 892/1808] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:23:47 2022] PUT /identity/v3/projects/350c2f8f5adb4b44992dd12b32d5d11e/users/158489cfd87e4d2cb82d07e45b641aed/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:47.803202 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38483c98-0bf6-4660-9037-cd15835c827b None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:47.803561 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38483c98-0bf6-4660-9037-cd15835c827b None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:47.803906 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38483c98-0bf6-4660-9037-cd15835c827b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:47.840407 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-38483c98-0bf6-4660-9037-cd15835c827b None None] MFA Rules not processed for user `158489cfd87e4d2cb82d07e45b641aed`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:47.841878 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-38483c98-0bf6-4660-9037-cd15835c827b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:47.914697 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:47.951117 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 917/1809] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:47 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:47.963616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85641ae-d18f-4ee6-b0c9-5a9b03d0e8cd admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:47.965742 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-b85641ae-d18f-4ee6-b0c9-5a9b03d0e8cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:48.053920 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85641ae-d18f-4ee6-b0c9-5a9b03d0e8cd admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:48.076147 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9527325-fbd1-467b-a454-a2a7d015ea21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:48.078370 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9527325-fbd1-467b-a454-a2a7d015ea21 service cinder] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:48.078657 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9527325-fbd1-467b-a454-a2a7d015ea21 service cinder] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:48.078935 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9527325-fbd1-467b-a454-a2a7d015ea21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:48.085635 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-d9527325-fbd1-467b-a454-a2a7d015ea21 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:48.247663 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9527325-fbd1-467b-a454-a2a7d015ea21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:48.249450 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9527325-fbd1-467b-a454-a2a7d015ea21 service cinder] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:48.299634 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 893/1810] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:48.658859 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38483c98-0bf6-4660-9037-cd15835c827b None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:48.661709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-38483c98-0bf6-4660-9037-cd15835c827b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:48.904235 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38483c98-0bf6-4660-9037-cd15835c827b None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:48.928470 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219e9958-e892-4e38-89f6-002dd093f7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:48.929468 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e9958-e892-4e38-89f6-002dd093f7d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:48.929749 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e9958-e892-4e38-89f6-002dd093f7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:48.944818 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e9958-e892-4e38-89f6-002dd093f7d9 admin admin] PATH_INFO: `/v3/projects/86f523489a104a25bbcd33215361c336` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:48.954872 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219e9958-e892-4e38-89f6-002dd093f7d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86f523489a104a25bbcd33215361c336)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:48.957621 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219e9958-e892-4e38-89f6-002dd093f7d9 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:49.155354 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-219e9958-e892-4e38-89f6-002dd093f7d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86f523489a104a25bbcd33215361c336', 'request_id': 'req-219e9958-e892-4e38-89f6-002dd093f7d9'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:49.177761 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-219e9958-e892-4e38-89f6-002dd093f7d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86f523489a104a25bbcd33215361c336', 'request_id': 'req-219e9958-e892-4e38-89f6-002dd093f7d9'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:49.216346 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-219e9958-e892-4e38-89f6-002dd093f7d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86f523489a104a25bbcd33215361c336', 'request_id': 'req-219e9958-e892-4e38-89f6-002dd093f7d9'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:49.248498 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-219e9958-e892-4e38-89f6-002dd093f7d9 admin admin] The token cache is being invalidate because project 86f523489a104a25bbcd33215361c336 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:23:49.248803 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-219e9958-e892-4e38-89f6-002dd093f7d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:49.257693 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-219e9958-e892-4e38-89f6-002dd093f7d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:49.262633 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 918/1811] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:23:48 2022] DELETE /identity/v3/projects/86f523489a104a25bbcd33215361c336 => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:49.268208 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9527325-fbd1-467b-a454-a2a7d015ea21 service cinder] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:49.290331 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9527325-fbd1-467b-a454-a2a7d015ea21 service cinder] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:49.506680 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d13aff-4a42-4ab1-b12d-db6a0de9d17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:49.507602 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d13aff-4a42-4ab1-b12d-db6a0de9d17b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:49.507929 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d13aff-4a42-4ab1-b12d-db6a0de9d17b admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:49.508272 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d13aff-4a42-4ab1-b12d-db6a0de9d17b admin admin] PATH_INFO: `/v3/users/053412d72e6e4b4e8865d50a1594a3b4` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:49.512322 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d13aff-4a42-4ab1-b12d-db6a0de9d17b admin admin] RBAC: Authorizing `identity:delete_user(user_id=053412d72e6e4b4e8865d50a1594a3b4)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:49.514738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d13aff-4a42-4ab1-b12d-db6a0de9d17b admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:49.653919 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-f6d13aff-4a42-4ab1-b12d-db6a0de9d17b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '053412d72e6e4b4e8865d50a1594a3b4'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:49.656996 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:49.690417 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-f6d13aff-4a42-4ab1-b12d-db6a0de9d17b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '053412d72e6e4b4e8865d50a1594a3b4'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:49.721845 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-f6d13aff-4a42-4ab1-b12d-db6a0de9d17b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '053412d72e6e4b4e8865d50a1594a3b4'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:49.764932 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-f6d13aff-4a42-4ab1-b12d-db6a0de9d17b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '053412d72e6e4b4e8865d50a1594a3b4'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:49.767497 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-f6d13aff-4a42-4ab1-b12d-db6a0de9d17b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '053412d72e6e4b4e8865d50a1594a3b4'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:49.778366 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655896d1-518c-4408-b484-b3a397734ece None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:49.778839 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655896d1-518c-4408-b484-b3a397734ece None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:49.778910 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655896d1-518c-4408-b484-b3a397734ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:49.791286 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 894/1812] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:23:49 2022] DELETE /identity/v3/users/053412d72e6e4b4e8865d50a1594a3b4 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:49.802853 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d636c-8ad9-49cb-bea5-72628c27571b None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:49.803366 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d636c-8ad9-49cb-bea5-72628c27571b None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:49.803690 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d636c-8ad9-49cb-bea5-72628c27571b None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:49.804361 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-655896d1-518c-4408-b484-b3a397734ece None None] MFA Rules not processed for user `a094273e734342eca2bdf159fab8a8d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:49.806571 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-655896d1-518c-4408-b484-b3a397734ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:49.808179 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 895/1813] 10.222.0.50 () {54 vars in 921 bytes} [Fri May 27 03:23:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:23:49.840559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8dac6e-2a5c-4108-b3b6-01beb504c52e None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:49.840853 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8dac6e-2a5c-4108-b3b6-01beb504c52e None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:49.841151 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8dac6e-2a5c-4108-b3b6-01beb504c52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:49.897601 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-eb8dac6e-2a5c-4108-b3b6-01beb504c52e None None] MFA Rules not processed for user `1a699988f08a4001b3b7d764f01f4fc5`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:49.898999 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-eb8dac6e-2a5c-4108-b3b6-01beb504c52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:49.912449 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 919/1814] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:50.069521 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 896/1815] 10.222.0.50 () {58 vars in 1018 bytes} [Fri May 27 03:23:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:23:50.408228 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655896d1-518c-4408-b484-b3a397734ece None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:50.410195 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-655896d1-518c-4408-b484-b3a397734ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:50.462522 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655896d1-518c-4408-b484-b3a397734ece None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:50.475185 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b37967-b2b3-4de8-92b2-121cdac03714 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:50.476080 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b37967-b2b3-4de8-92b2-121cdac03714 service nova] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:50.476503 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b37967-b2b3-4de8-92b2-121cdac03714 service nova] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:50.476990 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b37967-b2b3-4de8-92b2-121cdac03714 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:50.504172 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b37967-b2b3-4de8-92b2-121cdac03714 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:50.504172 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b37967-b2b3-4de8-92b2-121cdac03714 service nova] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:50.505975 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8dac6e-2a5c-4108-b3b6-01beb504c52e None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:50.517121 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-eb8dac6e-2a5c-4108-b3b6-01beb504c52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:50.550739 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 920/1816] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:50.646191 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8dac6e-2a5c-4108-b3b6-01beb504c52e None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:50.653626 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732d98f0-acf4-4fcb-95ed-0d9d2a8fea67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:50.654892 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d98f0-acf4-4fcb-95ed-0d9d2a8fea67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:50.655349 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d98f0-acf4-4fcb-95ed-0d9d2a8fea67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:50.655794 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d98f0-acf4-4fcb-95ed-0d9d2a8fea67 admin admin] PATH_INFO: `/v3/projects/6263657281eb42e691c38e6a90f57a9d` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:50.657435 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732d98f0-acf4-4fcb-95ed-0d9d2a8fea67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6263657281eb42e691c38e6a90f57a9d)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:50.660161 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732d98f0-acf4-4fcb-95ed-0d9d2a8fea67 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:50.680029 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b37967-b2b3-4de8-92b2-121cdac03714 service nova] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:50.681963 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-81b37967-b2b3-4de8-92b2-121cdac03714 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:50.690187 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:50.780811 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-732d98f0-acf4-4fcb-95ed-0d9d2a8fea67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6263657281eb42e691c38e6a90f57a9d', 'request_id': 'req-732d98f0-acf4-4fcb-95ed-0d9d2a8fea67'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:50.792095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-732d98f0-acf4-4fcb-95ed-0d9d2a8fea67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6263657281eb42e691c38e6a90f57a9d', 'request_id': 'req-732d98f0-acf4-4fcb-95ed-0d9d2a8fea67'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:50.802594 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:50.807254 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-732d98f0-acf4-4fcb-95ed-0d9d2a8fea67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6263657281eb42e691c38e6a90f57a9d', 'request_id': 'req-732d98f0-acf4-4fcb-95ed-0d9d2a8fea67'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:50.823480 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b37967-b2b3-4de8-92b2-121cdac03714 service nova] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:50.835476 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-732d98f0-acf4-4fcb-95ed-0d9d2a8fea67 admin admin] The token cache is being invalidate because project 6263657281eb42e691c38e6a90f57a9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:23:50.836119 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-732d98f0-acf4-4fcb-95ed-0d9d2a8fea67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:50.837910 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-732d98f0-acf4-4fcb-95ed-0d9d2a8fea67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:50.843199 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 897/1817] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:23:50 2022] DELETE /identity/v3/projects/6263657281eb42e691c38e6a90f57a9d => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:50.846111 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab30627a-cfbb-457a-b7e9-3b39acf34e44 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:50.849842 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab30627a-cfbb-457a-b7e9-3b39acf34e44 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:50.850567 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab30627a-cfbb-457a-b7e9-3b39acf34e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:50.910674 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-ab30627a-cfbb-457a-b7e9-3b39acf34e44 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:50.911899 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-ab30627a-cfbb-457a-b7e9-3b39acf34e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:51.005676 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb879e-858f-462b-aa3e-40fb9ce03eec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:51.006470 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb879e-858f-462b-aa3e-40fb9ce03eec service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:51.006755 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb879e-858f-462b-aa3e-40fb9ce03eec service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:51.008312 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb879e-858f-462b-aa3e-40fb9ce03eec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:51.018328 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-4bbb879e-858f-462b-aa3e-40fb9ce03eec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:51.078737 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 898/1818] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:51.088515 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9035c1b-2f46-4928-8f76-4e04cc9c4709 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:51.089383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9035c1b-2f46-4928-8f76-4e04cc9c4709 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:51.089876 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9035c1b-2f46-4928-8f76-4e04cc9c4709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:51.119898 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-c9035c1b-2f46-4928-8f76-4e04cc9c4709 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:51.121969 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-c9035c1b-2f46-4928-8f76-4e04cc9c4709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:51.134956 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 899/1819] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:51.142914 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9035c1b-2f46-4928-8f76-4e04cc9c4709 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:51.171482 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9035c1b-2f46-4928-8f76-4e04cc9c4709 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:51.187625 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483d0c7-e58a-4602-a163-960aa79620e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:51.188591 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483d0c7-e58a-4602-a163-960aa79620e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:51.188963 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483d0c7-e58a-4602-a163-960aa79620e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:51.189315 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9483d0c7-e58a-4602-a163-960aa79620e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:51.190131 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483d0c7-e58a-4602-a163-960aa79620e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:23:51.190506 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483d0c7-e58a-4602-a163-960aa79620e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:51.192362 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb879e-858f-462b-aa3e-40fb9ce03eec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:51.193520 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9483d0c7-e58a-4602-a163-960aa79620e6 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:51.196583 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb879e-858f-462b-aa3e-40fb9ce03eec service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:51.217425 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 921/1820] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:23:51.230339 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 900/1821] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:23:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:23:51.231127 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb879e-858f-462b-aa3e-40fb9ce03eec service cinder] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:51.245002 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb879e-858f-462b-aa3e-40fb9ce03eec service cinder] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:51.250850 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcbd07e-b7b7-43cc-a19b-ae0418d92d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:51.251798 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbd07e-b7b7-43cc-a19b-ae0418d92d8e admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:51.252213 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbd07e-b7b7-43cc-a19b-ae0418d92d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:51.252673 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcbd07e-b7b7-43cc-a19b-ae0418d92d8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:51.253605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcbd07e-b7b7-43cc-a19b-ae0418d92d8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:51.256077 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcbd07e-b7b7-43cc-a19b-ae0418d92d8e admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:51.306883 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483d0c7-e58a-4602-a163-960aa79620e6 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:51.320071 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 922/1822] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:23:51 2022] POST /identity/v3/projects => generated 387 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:51.323537 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9483d0c7-e58a-4602-a163-960aa79620e6 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:51.436614 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94edf4-54f7-4358-a5ca-ce15b67685ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:51.437724 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94edf4-54f7-4358-a5ca-ce15b67685ef admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:51.438485 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94edf4-54f7-4358-a5ca-ce15b67685ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:51.438955 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94edf4-54f7-4358-a5ca-ce15b67685ef admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:51.440337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb94edf4-54f7-4358-a5ca-ce15b67685ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:51.443742 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb94edf4-54f7-4358-a5ca-ce15b67685ef admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:51.487465 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 901/1823] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:23:51 2022] POST /identity/v3/users => generated 294 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:51.492896 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcbd07e-b7b7-43cc-a19b-ae0418d92d8e admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:51.508288 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcbd07e-b7b7-43cc-a19b-ae0418d92d8e admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:51.518625 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e022b550-8c2d-48b8-a83c-64337e035e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:51.520489 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e022b550-8c2d-48b8-a83c-64337e035e86 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:51.520863 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e022b550-8c2d-48b8-a83c-64337e035e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:51.521258 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e022b550-8c2d-48b8-a83c-64337e035e86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:51.522295 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e022b550-8c2d-48b8-a83c-64337e035e86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:51.525550 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e022b550-8c2d-48b8-a83c-64337e035e86 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:51.546716 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94edf4-54f7-4358-a5ca-ce15b67685ef admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:51.557207 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 923/1824] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:51 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:51.563362 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb94edf4-54f7-4358-a5ca-ce15b67685ef admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:51.571031 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25fa5fa-b5f9-4500-b92e-ea59032ff825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:51.571956 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fa5fa-b5f9-4500-b92e-ea59032ff825 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:51.572516 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fa5fa-b5f9-4500-b92e-ea59032ff825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:51.573070 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fa5fa-b5f9-4500-b92e-ea59032ff825 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:51.574327 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25fa5fa-b5f9-4500-b92e-ea59032ff825 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:51.576403 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25fa5fa-b5f9-4500-b92e-ea59032ff825 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:51.600552 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e022b550-8c2d-48b8-a83c-64337e035e86 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:51.613222 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 902/1825] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:51 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:51.615019 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e022b550-8c2d-48b8-a83c-64337e035e86 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:51.621777 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b8f07-d93c-4b2c-9828-286e5d163c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:51.622905 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b8f07-d93c-4b2c-9828-286e5d163c30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:51.623279 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b8f07-d93c-4b2c-9828-286e5d163c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:51.623709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b8f07-d93c-4b2c-9828-286e5d163c30 admin admin] PATH_INFO: `/v3/projects/233d1aa4475e41d2b56c628ab850fd79/users/a6d836e92a114a469241b4837703d5c4/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:51.645200 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1b8f07-d93c-4b2c-9828-286e5d163c30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=233d1aa4475e41d2b56c628ab850fd79, user_id=a6d836e92a114a469241b4837703d5c4, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:51.649196 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1b8f07-d93c-4b2c-9828-286e5d163c30 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:51.704331 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 924/1826] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:23:51 2022] PUT /identity/v3/projects/233d1aa4475e41d2b56c628ab850fd79/users/a6d836e92a114a469241b4837703d5c4/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:51.708749 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eba237-78df-4561-b6da-8395d1d76c7d None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:51.709293 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eba237-78df-4561-b6da-8395d1d76c7d None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:51.709781 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eba237-78df-4561-b6da-8395d1d76c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:51.757198 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-e7eba237-78df-4561-b6da-8395d1d76c7d None None] MFA Rules not processed for user `a6d836e92a114a469241b4837703d5c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:51.759677 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-e7eba237-78df-4561-b6da-8395d1d76c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:51.811916 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:51.893611 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 903/1827] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:51.899081 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b8f07-d93c-4b2c-9828-286e5d163c30 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:51.928126 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:51.996213 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b8f07-d93c-4b2c-9828-286e5d163c30 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:52.000774 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df319a8-914f-47f4-ab21-5fbfa7117996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:52.001451 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df319a8-914f-47f4-ab21-5fbfa7117996 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:52.001883 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df319a8-914f-47f4-ab21-5fbfa7117996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:52.002150 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df319a8-914f-47f4-ab21-5fbfa7117996 admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:52.003014 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df319a8-914f-47f4-ab21-5fbfa7117996 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:52.005124 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df319a8-914f-47f4-ab21-5fbfa7117996 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:52.036099 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 925/1828] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:23:51 2022] POST /identity/v3/projects => generated 383 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:23:52.040326 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eba237-78df-4561-b6da-8395d1d76c7d None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:52.056144 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eba237-78df-4561-b6da-8395d1d76c7d None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:52.133410 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636974c-468b-4b02-8e52-deab8085af01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:52.134235 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636974c-468b-4b02-8e52-deab8085af01 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:52.134585 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636974c-468b-4b02-8e52-deab8085af01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:52.134934 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636974c-468b-4b02-8e52-deab8085af01 admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:52.135707 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7636974c-468b-4b02-8e52-deab8085af01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:52.137776 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7636974c-468b-4b02-8e52-deab8085af01 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:52.165892 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 904/1829] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:23:52 2022] POST /identity/v3/users => generated 298 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:23:52.167357 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df319a8-914f-47f4-ab21-5fbfa7117996 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:52.184662 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df319a8-914f-47f4-ab21-5fbfa7117996 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:52.194635 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2144aa-6c6c-40e8-a383-02dcd5d37531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:52.195611 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2144aa-6c6c-40e8-a383-02dcd5d37531 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:52.196136 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2144aa-6c6c-40e8-a383-02dcd5d37531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:52.196760 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2144aa-6c6c-40e8-a383-02dcd5d37531 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:52.197826 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2144aa-6c6c-40e8-a383-02dcd5d37531 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:52.199882 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2144aa-6c6c-40e8-a383-02dcd5d37531 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:52.228859 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 926/1830] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:23:52.233304 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636974c-468b-4b02-8e52-deab8085af01 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:52.244340 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636974c-468b-4b02-8e52-deab8085af01 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:52.250368 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1577d089-3ee6-4d67-80e0-29ce80a1f3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:52.251528 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577d089-3ee6-4d67-80e0-29ce80a1f3e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:52.251956 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577d089-3ee6-4d67-80e0-29ce80a1f3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:52.252373 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577d089-3ee6-4d67-80e0-29ce80a1f3e3 admin admin] PATH_INFO: `/v3/projects/0edfc217597c4f3fa1206e7ff4ff479b/users/47963df171724d038e4590d284e94173/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:52.269368 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1577d089-3ee6-4d67-80e0-29ce80a1f3e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0edfc217597c4f3fa1206e7ff4ff479b, user_id=47963df171724d038e4590d284e94173, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:52.272073 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1577d089-3ee6-4d67-80e0-29ce80a1f3e3 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:52.290728 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 905/1831] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:23:52 2022] PUT /identity/v3/projects/0edfc217597c4f3fa1206e7ff4ff479b/users/47963df171724d038e4590d284e94173/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:52.298391 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2144aa-6c6c-40e8-a383-02dcd5d37531 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:52.311804 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2144aa-6c6c-40e8-a383-02dcd5d37531 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:52.395947 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0832f8c3-fb1d-433e-a75a-5fe3306ecfe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:52.396633 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832f8c3-fb1d-433e-a75a-5fe3306ecfe7 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:52.396916 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832f8c3-fb1d-433e-a75a-5fe3306ecfe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:52.397227 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832f8c3-fb1d-433e-a75a-5fe3306ecfe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:52.397957 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0832f8c3-fb1d-433e-a75a-5fe3306ecfe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:52.399793 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0832f8c3-fb1d-433e-a75a-5fe3306ecfe7 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:52.410246 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 927/1832] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:52.414717 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1577d089-3ee6-4d67-80e0-29ce80a1f3e3 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:52.428700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1577d089-3ee6-4d67-80e0-29ce80a1f3e3 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:52.434621 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4892bdd-c0b4-402b-999f-a9e75114f660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:52.436007 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4892bdd-c0b4-402b-999f-a9e75114f660 admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:52.436348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4892bdd-c0b4-402b-999f-a9e75114f660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:52.436880 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4892bdd-c0b4-402b-999f-a9e75114f660 admin admin] PATH_INFO: `/v3/projects/0edfc217597c4f3fa1206e7ff4ff479b/users/47963df171724d038e4590d284e94173/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:52.439327 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4892bdd-c0b4-402b-999f-a9e75114f660 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0edfc217597c4f3fa1206e7ff4ff479b, user_id=47963df171724d038e4590d284e94173, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:52.442532 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4892bdd-c0b4-402b-999f-a9e75114f660 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:52.456905 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 906/1833] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:23:52 2022] PUT /identity/v3/projects/0edfc217597c4f3fa1206e7ff4ff479b/users/47963df171724d038e4590d284e94173/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:52.475162 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29becf-5326-425f-99d8-ba4dced51c6f None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:52.475447 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29becf-5326-425f-99d8-ba4dced51c6f None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:52.475718 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29becf-5326-425f-99d8-ba4dced51c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:52.506167 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-2f29becf-5326-425f-99d8-ba4dced51c6f None None] MFA Rules not processed for user `47963df171724d038e4590d284e94173`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:52.507379 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-2f29becf-5326-425f-99d8-ba4dced51c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:52.577898 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 928/1834] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:52 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:52.589457 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4892bdd-c0b4-402b-999f-a9e75114f660 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:52.591490 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-c4892bdd-c0b4-402b-999f-a9e75114f660 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:52.654672 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4892bdd-c0b4-402b-999f-a9e75114f660 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:52.658319 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939339a9-7b92-4dc5-8dac-afc721fbb134 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:52.658955 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939339a9-7b92-4dc5-8dac-afc721fbb134 service cinder] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:52.659234 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939339a9-7b92-4dc5-8dac-afc721fbb134 service cinder] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:52.659509 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939339a9-7b92-4dc5-8dac-afc721fbb134 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:52.669651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939339a9-7b92-4dc5-8dac-afc721fbb134 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:52.671096 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939339a9-7b92-4dc5-8dac-afc721fbb134 service cinder] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:52.680652 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 907/1835] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:52 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:52.709519 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29becf-5326-425f-99d8-ba4dced51c6f None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:52.724399 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:52.777102 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29becf-5326-425f-99d8-ba4dced51c6f None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:52.781972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5d5b3-4dfb-4230-b738-f567b4dc4ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:52.782864 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5d5b3-4dfb-4230-b738-f567b4dc4ed3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:52.783145 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5d5b3-4dfb-4230-b738-f567b4dc4ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:52.783425 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5d5b3-4dfb-4230-b738-f567b4dc4ed3 admin admin] PATH_INFO: `/v3/users/a6d836e92a114a469241b4837703d5c4` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:52.784651 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b5d5b3-4dfb-4230-b738-f567b4dc4ed3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6d836e92a114a469241b4837703d5c4)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:52.786644 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b5d5b3-4dfb-4230-b738-f567b4dc4ed3 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:52.836177 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-43b5d5b3-4dfb-4230-b738-f567b4dc4ed3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6d836e92a114a469241b4837703d5c4'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:52.837004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-43b5d5b3-4dfb-4230-b738-f567b4dc4ed3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6d836e92a114a469241b4837703d5c4'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:52.846006 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-43b5d5b3-4dfb-4230-b738-f567b4dc4ed3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6d836e92a114a469241b4837703d5c4'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:52.846622 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-43b5d5b3-4dfb-4230-b738-f567b4dc4ed3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6d836e92a114a469241b4837703d5c4'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:52.859197 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-43b5d5b3-4dfb-4230-b738-f567b4dc4ed3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6d836e92a114a469241b4837703d5c4'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:52.873219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 929/1836] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:23:52 2022] DELETE /identity/v3/users/a6d836e92a114a469241b4837703d5c4 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:52.877676 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5e14e-965c-47ce-b602-cb5c464c8670 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:52.878542 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5e14e-965c-47ce-b602-cb5c464c8670 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:52.878542 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c5e14e-965c-47ce-b602-cb5c464c8670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:52.898352 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-60c5e14e-965c-47ce-b602-cb5c464c8670 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:52.899644 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-60c5e14e-965c-47ce-b602-cb5c464c8670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:52.930994 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:52.983358 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 908/1837] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:52.993373 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5d5b3-4dfb-4230-b738-f567b4dc4ed3 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:52.994878 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-43b5d5b3-4dfb-4230-b738-f567b4dc4ed3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:53.187640 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5d5b3-4dfb-4230-b738-f567b4dc4ed3 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:53.191553 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d657ec1-1507-4c2a-8c17-b1e0676fe44f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:53.192201 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d657ec1-1507-4c2a-8c17-b1e0676fe44f service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:53.192537 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d657ec1-1507-4c2a-8c17-b1e0676fe44f service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:53.192828 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d657ec1-1507-4c2a-8c17-b1e0676fe44f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:53.202707 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d657ec1-1507-4c2a-8c17-b1e0676fe44f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:53.204199 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d657ec1-1507-4c2a-8c17-b1e0676fe44f service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:53.214111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 930/1838] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:53.529460 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5e14e-965c-47ce-b602-cb5c464c8670 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:53.531822 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-60c5e14e-965c-47ce-b602-cb5c464c8670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:53.557326 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c5e14e-965c-47ce-b602-cb5c464c8670 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:53.562108 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cefa25-9624-4093-b606-0cd5b93d3b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:53.562987 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cefa25-9624-4093-b606-0cd5b93d3b46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:53.563261 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cefa25-9624-4093-b606-0cd5b93d3b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:53.563530 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cefa25-9624-4093-b606-0cd5b93d3b46 admin admin] PATH_INFO: `/v3/projects/233d1aa4475e41d2b56c628ab850fd79` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:53.564719 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cefa25-9624-4093-b606-0cd5b93d3b46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=233d1aa4475e41d2b56c628ab850fd79)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:53.566665 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cefa25-9624-4093-b606-0cd5b93d3b46 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:53.590609 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d657ec1-1507-4c2a-8c17-b1e0676fe44f service neutron] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:53.592684 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-5d657ec1-1507-4c2a-8c17-b1e0676fe44f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:53.626185 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-a9cefa25-9624-4093-b606-0cd5b93d3b46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '233d1aa4475e41d2b56c628ab850fd79', 'request_id': 'req-a9cefa25-9624-4093-b606-0cd5b93d3b46'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:53.637380 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-a9cefa25-9624-4093-b606-0cd5b93d3b46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '233d1aa4475e41d2b56c628ab850fd79', 'request_id': 'req-a9cefa25-9624-4093-b606-0cd5b93d3b46'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:53.647011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-a9cefa25-9624-4093-b606-0cd5b93d3b46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '233d1aa4475e41d2b56c628ab850fd79', 'request_id': 'req-a9cefa25-9624-4093-b606-0cd5b93d3b46'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:53.652182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:53.669649 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-a9cefa25-9624-4093-b606-0cd5b93d3b46 admin admin] The token cache is being invalidate because project 233d1aa4475e41d2b56c628ab850fd79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:23:53.670104 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-a9cefa25-9624-4093-b606-0cd5b93d3b46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:53.670895 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-a9cefa25-9624-4093-b606-0cd5b93d3b46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:53.673639 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 909/1839] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:23:53 2022] DELETE /identity/v3/projects/233d1aa4475e41d2b56c628ab850fd79 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:53.676064 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d657ec1-1507-4c2a-8c17-b1e0676fe44f service neutron] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:53.677684 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cefa25-9624-4093-b606-0cd5b93d3b46 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:53.680470 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-a9cefa25-9624-4093-b606-0cd5b93d3b46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:53.744943 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d87771-ddba-42d5-9492-152f30daacef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:53.745783 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d87771-ddba-42d5-9492-152f30daacef service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:53.746191 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d87771-ddba-42d5-9492-152f30daacef service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:53.746470 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d87771-ddba-42d5-9492-152f30daacef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:53.748503 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-d3d87771-ddba-42d5-9492-152f30daacef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:53.757938 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cefa25-9624-4093-b606-0cd5b93d3b46 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:53.763215 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd823a-985d-4f41-9b01-2b3daf4ee1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:53.764175 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd823a-985d-4f41-9b01-2b3daf4ee1f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:53.764494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd823a-985d-4f41-9b01-2b3daf4ee1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:53.765815 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd823a-985d-4f41-9b01-2b3daf4ee1f9 admin admin] PATH_INFO: `/v3/users/47963df171724d038e4590d284e94173` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:53.770745 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acd823a-985d-4f41-9b01-2b3daf4ee1f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47963df171724d038e4590d284e94173)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:53.772747 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acd823a-985d-4f41-9b01-2b3daf4ee1f9 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:53.821221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d87771-ddba-42d5-9492-152f30daacef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:53.822682 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d87771-ddba-42d5-9492-152f30daacef service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:53.828615 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-7acd823a-985d-4f41-9b01-2b3daf4ee1f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47963df171724d038e4590d284e94173'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:53.833942 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 931/1840] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:53 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:23:53.836080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:53.842708 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-7acd823a-985d-4f41-9b01-2b3daf4ee1f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47963df171724d038e4590d284e94173'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:53.851205 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-7acd823a-985d-4f41-9b01-2b3daf4ee1f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47963df171724d038e4590d284e94173'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:53.860145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-7acd823a-985d-4f41-9b01-2b3daf4ee1f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47963df171724d038e4590d284e94173'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:53.861116 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-7acd823a-985d-4f41-9b01-2b3daf4ee1f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47963df171724d038e4590d284e94173'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:53.869863 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 910/1841] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:23:53 2022] DELETE /identity/v3/users/47963df171724d038e4590d284e94173 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:54.223049 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d87771-ddba-42d5-9492-152f30daacef service cinder] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:54.235186 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d87771-ddba-42d5-9492-152f30daacef service cinder] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:54.306602 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a73d5de-28e9-44fb-bf89-23d2c25990cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:54.307490 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73d5de-28e9-44fb-bf89-23d2c25990cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:54.307769 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73d5de-28e9-44fb-bf89-23d2c25990cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:54.308041 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a73d5de-28e9-44fb-bf89-23d2c25990cd admin admin] PATH_INFO: `/v3/projects/0edfc217597c4f3fa1206e7ff4ff479b` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:54.309298 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a73d5de-28e9-44fb-bf89-23d2c25990cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=0edfc217597c4f3fa1206e7ff4ff479b)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:54.311230 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a73d5de-28e9-44fb-bf89-23d2c25990cd admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:54.367325 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:54.371089 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3a73d5de-28e9-44fb-bf89-23d2c25990cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0edfc217597c4f3fa1206e7ff4ff479b', 'request_id': 'req-3a73d5de-28e9-44fb-bf89-23d2c25990cd'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:54.377900 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3a73d5de-28e9-44fb-bf89-23d2c25990cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0edfc217597c4f3fa1206e7ff4ff479b', 'request_id': 'req-3a73d5de-28e9-44fb-bf89-23d2c25990cd'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:54.386998 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3a73d5de-28e9-44fb-bf89-23d2c25990cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0edfc217597c4f3fa1206e7ff4ff479b', 'request_id': 'req-3a73d5de-28e9-44fb-bf89-23d2c25990cd'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:54.400719 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3a73d5de-28e9-44fb-bf89-23d2c25990cd admin admin] The token cache is being invalidate because project 0edfc217597c4f3fa1206e7ff4ff479b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:23:54.401113 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3a73d5de-28e9-44fb-bf89-23d2c25990cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:54.401839 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3a73d5de-28e9-44fb-bf89-23d2c25990cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:54.408127 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 932/1842] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:23:54 2022] DELETE /identity/v3/projects/0edfc217597c4f3fa1206e7ff4ff479b => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:54.409829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeefd671-b425-44be-a62e-c53c83b7680c None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:54.410115 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeefd671-b425-44be-a62e-c53c83b7680c None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:54.410384 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeefd671-b425-44be-a62e-c53c83b7680c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:54.429311 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-aeefd671-b425-44be-a62e-c53c83b7680c None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:54.430969 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-aeefd671-b425-44be-a62e-c53c83b7680c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:54.495934 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 911/1843] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:23:54.504574 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb022e34-05bd-4d7e-8f9a-f2a4c14832a6 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:54.505344 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb022e34-05bd-4d7e-8f9a-f2a4c14832a6 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:54.505833 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb022e34-05bd-4d7e-8f9a-f2a4c14832a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:54.531106 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-fb022e34-05bd-4d7e-8f9a-f2a4c14832a6 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:54.532480 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-fb022e34-05bd-4d7e-8f9a-f2a4c14832a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:54.548174 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 933/1844] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:54.552636 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeefd671-b425-44be-a62e-c53c83b7680c None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:54.563438 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeefd671-b425-44be-a62e-c53c83b7680c None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:54.568015 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676aa1ac-0a86-4b48-895b-e1364b0fd783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:54.568689 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676aa1ac-0a86-4b48-895b-e1364b0fd783 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:54.569162 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676aa1ac-0a86-4b48-895b-e1364b0fd783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:54.569448 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676aa1ac-0a86-4b48-895b-e1364b0fd783 admin admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:54.570197 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676aa1ac-0a86-4b48-895b-e1364b0fd783 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:23:54.570636 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676aa1ac-0a86-4b48-895b-e1364b0fd783 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:54.572389 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676aa1ac-0a86-4b48-895b-e1364b0fd783 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:54.585806 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 912/1845] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:23:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:23:54.590582 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb022e34-05bd-4d7e-8f9a-f2a4c14832a6 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:54.600758 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb022e34-05bd-4d7e-8f9a-f2a4c14832a6 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:54.605871 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e968fc4-62af-488d-9a1e-0a52987d052f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:54.607466 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e968fc4-62af-488d-9a1e-0a52987d052f admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:54.607466 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e968fc4-62af-488d-9a1e-0a52987d052f admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:54.607466 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e968fc4-62af-488d-9a1e-0a52987d052f admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:54.607893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e968fc4-62af-488d-9a1e-0a52987d052f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:54.609831 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e968fc4-62af-488d-9a1e-0a52987d052f admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:54.668343 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 934/1846] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:23:54 2022] POST /identity/v3/projects => generated 375 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:23:54.672688 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676aa1ac-0a86-4b48-895b-e1364b0fd783 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:54.686240 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676aa1ac-0a86-4b48-895b-e1364b0fd783 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:54.846681 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13184e95-0487-461b-bfc0-a39a9f45fb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:54.846681 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13184e95-0487-461b-bfc0-a39a9f45fb81 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:54.846681 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13184e95-0487-461b-bfc0-a39a9f45fb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:54.846681 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13184e95-0487-461b-bfc0-a39a9f45fb81 admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:54.847116 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13184e95-0487-461b-bfc0-a39a9f45fb81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:54.847116 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13184e95-0487-461b-bfc0-a39a9f45fb81 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:54.877513 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e968fc4-62af-488d-9a1e-0a52987d052f admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:54.885059 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 913/1847] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:23:54 2022] POST /identity/v3/users => generated 288 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:23:54.891530 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e968fc4-62af-488d-9a1e-0a52987d052f admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:54.899481 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d33aa-71bf-4632-bfc7-95c04af45e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:54.900273 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d33aa-71bf-4632-bfc7-95c04af45e3f admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:54.900671 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d33aa-71bf-4632-bfc7-95c04af45e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:54.901119 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121d33aa-71bf-4632-bfc7-95c04af45e3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:54.901953 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121d33aa-71bf-4632-bfc7-95c04af45e3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:54.904013 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121d33aa-71bf-4632-bfc7-95c04af45e3f admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:54.923715 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 935/1848] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:54 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:54.934037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13184e95-0487-461b-bfc0-a39a9f45fb81 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:54.950012 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13184e95-0487-461b-bfc0-a39a9f45fb81 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:54.955258 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3129f0-ddd6-4297-bd97-2b4074909d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:54.955944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3129f0-ddd6-4297-bd97-2b4074909d30 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:54.956230 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3129f0-ddd6-4297-bd97-2b4074909d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:54.956501 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3129f0-ddd6-4297-bd97-2b4074909d30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:54.957301 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3129f0-ddd6-4297-bd97-2b4074909d30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:54.959038 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3129f0-ddd6-4297-bd97-2b4074909d30 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:54.971693 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 914/1849] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:23:54 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:23:54.975655 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d33aa-71bf-4632-bfc7-95c04af45e3f admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:54.987245 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121d33aa-71bf-4632-bfc7-95c04af45e3f admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:54.995088 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ef39f-8830-44d9-a60b-02eee6550222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:55.004448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ef39f-8830-44d9-a60b-02eee6550222 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:55.004448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ef39f-8830-44d9-a60b-02eee6550222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:55.004448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ef39f-8830-44d9-a60b-02eee6550222 admin admin] PATH_INFO: `/v3/projects/53f6e32b4ad44341ba7d0cb848ef6bc5/users/19e6611ed2f74ea3b9d053aae16257fe/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:55.026739 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70ef39f-8830-44d9-a60b-02eee6550222 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53f6e32b4ad44341ba7d0cb848ef6bc5, user_id=19e6611ed2f74ea3b9d053aae16257fe, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:55.031063 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70ef39f-8830-44d9-a60b-02eee6550222 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:55.070637 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 936/1850] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:23:54 2022] PUT /identity/v3/projects/53f6e32b4ad44341ba7d0cb848ef6bc5/users/19e6611ed2f74ea3b9d053aae16257fe/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:55.071228 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f2565-3e10-48ef-8139-9095575f6059 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:55.071589 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f2565-3e10-48ef-8139-9095575f6059 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:55.071911 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f2565-3e10-48ef-8139-9095575f6059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:55.136476 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-975f2565-3e10-48ef-8139-9095575f6059 None None] MFA Rules not processed for user `19e6611ed2f74ea3b9d053aae16257fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:55.138223 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-975f2565-3e10-48ef-8139-9095575f6059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:55.219497 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:55.302911 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 915/1851] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:23:55.320406 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ef39f-8830-44d9-a60b-02eee6550222 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:55.322754 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-f70ef39f-8830-44d9-a60b-02eee6550222 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:55.430183 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ef39f-8830-44d9-a60b-02eee6550222 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:55.434553 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36022c-8653-46dd-9220-694fa01e63f7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3c9c5c8d2e4376b1b70d974d4cb851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:55.435327 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36022c-8653-46dd-9220-694fa01e63f7 service glance] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:55.436175 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36022c-8653-46dd-9220-694fa01e63f7 service glance] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:55.436459 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36022c-8653-46dd-9220-694fa01e63f7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:55.448097 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c36022c-8653-46dd-9220-694fa01e63f7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:55.449811 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c36022c-8653-46dd-9220-694fa01e63f7 service glance] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:55.460582 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 937/1852] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:58.406964 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f2565-3e10-48ef-8139-9095575f6059 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:58.408423 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-975f2565-3e10-48ef-8139-9095575f6059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:58.468536 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36022c-8653-46dd-9220-694fa01e63f7 service glance] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:58.470380 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-5c36022c-8653-46dd-9220-694fa01e63f7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:58.597403 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f2565-3e10-48ef-8139-9095575f6059 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:58.611275 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa65c1-96f9-4fcb-9ce2-bcbc39011d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:58.612896 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa65c1-96f9-4fcb-9ce2-bcbc39011d66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:58.613481 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa65c1-96f9-4fcb-9ce2-bcbc39011d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:58.613894 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa65c1-96f9-4fcb-9ce2-bcbc39011d66 admin admin] PATH_INFO: `/v3/users/91978f5026db4454adba2bf18ecd917b` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:58.615428 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa65c1-96f9-4fcb-9ce2-bcbc39011d66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91978f5026db4454adba2bf18ecd917b)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:58.618247 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa65c1-96f9-4fcb-9ce2-bcbc39011d66 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:58.639489 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36022c-8653-46dd-9220-694fa01e63f7 service glance] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:58.665708 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788017cc-ad78-43be-9107-7b9386e93d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:58.666880 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788017cc-ad78-43be-9107-7b9386e93d3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:58.667474 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788017cc-ad78-43be-9107-7b9386e93d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:58.667872 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788017cc-ad78-43be-9107-7b9386e93d3c admin admin] PATH_INFO: `/v3/users/235dd34113d043289d44e51419838b11` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:58.671033 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788017cc-ad78-43be-9107-7b9386e93d3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=235dd34113d043289d44e51419838b11)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:58.673305 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788017cc-ad78-43be-9107-7b9386e93d3c admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:58.699556 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:58.737705 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-c6fa65c1-96f9-4fcb-9ce2-bcbc39011d66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91978f5026db4454adba2bf18ecd917b'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:58.777159 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-c6fa65c1-96f9-4fcb-9ce2-bcbc39011d66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91978f5026db4454adba2bf18ecd917b'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:58.792051 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-788017cc-ad78-43be-9107-7b9386e93d3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '235dd34113d043289d44e51419838b11'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:58.793194 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-788017cc-ad78-43be-9107-7b9386e93d3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '235dd34113d043289d44e51419838b11'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:58.799452 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-c6fa65c1-96f9-4fcb-9ce2-bcbc39011d66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91978f5026db4454adba2bf18ecd917b'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:58.810527 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-c6fa65c1-96f9-4fcb-9ce2-bcbc39011d66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91978f5026db4454adba2bf18ecd917b'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:58.812307 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-c6fa65c1-96f9-4fcb-9ce2-bcbc39011d66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91978f5026db4454adba2bf18ecd917b'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:58.813616 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-788017cc-ad78-43be-9107-7b9386e93d3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '235dd34113d043289d44e51419838b11'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:58.814453 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-788017cc-ad78-43be-9107-7b9386e93d3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '235dd34113d043289d44e51419838b11'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:58.822170 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 916/1853] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:23:58 2022] DELETE /identity/v3/users/91978f5026db4454adba2bf18ecd917b => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:23:58.827690 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983241f3-5353-473a-bd8c-1348bd0aa240 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:58.828242 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983241f3-5353-473a-bd8c-1348bd0aa240 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:58.829068 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983241f3-5353-473a-bd8c-1348bd0aa240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:58.831429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-788017cc-ad78-43be-9107-7b9386e93d3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '235dd34113d043289d44e51419838b11'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:23:58.865457 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-983241f3-5353-473a-bd8c-1348bd0aa240 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:58.866994 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-983241f3-5353-473a-bd8c-1348bd0aa240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:58.873150 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 938/1854] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:23:58 2022] DELETE /identity/v3/users/235dd34113d043289d44e51419838b11 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:23:58.882540 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0257c5ee-0cf6-4e6b-adf1-975335907557 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:58.882906 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0257c5ee-0cf6-4e6b-adf1-975335907557 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:58.883247 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0257c5ee-0cf6-4e6b-adf1-975335907557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:58.916633 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-0257c5ee-0cf6-4e6b-adf1-975335907557 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:23:58.919585 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-0257c5ee-0cf6-4e6b-adf1-975335907557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:58.950227 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:23:58.986560 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 917/1855] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:59.008342 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983241f3-5353-473a-bd8c-1348bd0aa240 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:59.009952 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-983241f3-5353-473a-bd8c-1348bd0aa240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:59.029344 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 939/1856] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:23:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:23:59.049650 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0257c5ee-0cf6-4e6b-adf1-975335907557 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:23:59.053586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-0257c5ee-0cf6-4e6b-adf1-975335907557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:23:59.176581 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0257c5ee-0cf6-4e6b-adf1-975335907557 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:59.181811 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784fce08-28df-4980-9941-973b285438fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:59.182923 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fce08-28df-4980-9941-973b285438fd service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:59.183281 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fce08-28df-4980-9941-973b285438fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:59.183803 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784fce08-28df-4980-9941-973b285438fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:59.202951 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983241f3-5353-473a-bd8c-1348bd0aa240 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:23:59.204253 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784fce08-28df-4980-9941-973b285438fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:59.206511 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784fce08-28df-4980-9941-973b285438fd service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:59.213435 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d67432f-99b6-49e9-9c6d-42c4412476e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:23:59.214097 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67432f-99b6-49e9-9c6d-42c4412476e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:23:59.214376 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67432f-99b6-49e9-9c6d-42c4412476e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:23:59.214643 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d67432f-99b6-49e9-9c6d-42c4412476e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:23:59.219887 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 940/1857] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:23:59.224808 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d67432f-99b6-49e9-9c6d-42c4412476e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:23:59.226488 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d67432f-99b6-49e9-9c6d-42c4412476e1 service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:23:59.241920 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 918/1858] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:23:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:24:00.053175 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784fce08-28df-4980-9941-973b285438fd service neutron] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:00.055004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-784fce08-28df-4980-9941-973b285438fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:00.073704 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784fce08-28df-4980-9941-973b285438fd service neutron] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:00.092425 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d67432f-99b6-49e9-9c6d-42c4412476e1 service neutron] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:00.096613 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-8d67432f-99b6-49e9-9c6d-42c4412476e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:00.112637 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c1b6d-7c0a-4c19-b28f-17b91e78d739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:00.121454 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c1b6d-7c0a-4c19-b28f-17b91e78d739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:00.123405 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c1b6d-7c0a-4c19-b28f-17b91e78d739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:00.124894 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c1b6d-7c0a-4c19-b28f-17b91e78d739 admin admin] PATH_INFO: `/v3/projects/1c488f6c9420498281252528d879a29b` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:00.126527 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5c1b6d-7c0a-4c19-b28f-17b91e78d739 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c488f6c9420498281252528d879a29b)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:00.140999 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d67432f-99b6-49e9-9c6d-42c4412476e1 service neutron] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:00.141525 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5c1b6d-7c0a-4c19-b28f-17b91e78d739 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:00.163291 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f72c5-e725-44d8-b652-94613bd8137f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:00.164700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f72c5-e725-44d8-b652-94613bd8137f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:00.165373 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f72c5-e725-44d8-b652-94613bd8137f admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:00.165808 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f72c5-e725-44d8-b652-94613bd8137f admin admin] PATH_INFO: `/v3/projects/8f95e0c579a7421cbe237f5715de337d` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:00.168260 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647f72c5-e725-44d8-b652-94613bd8137f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f95e0c579a7421cbe237f5715de337d)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:00.171146 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647f72c5-e725-44d8-b652-94613bd8137f admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:00.266015 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:00.284735 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-647f72c5-e725-44d8-b652-94613bd8137f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f95e0c579a7421cbe237f5715de337d', 'request_id': 'req-647f72c5-e725-44d8-b652-94613bd8137f'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:00.308337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-647f72c5-e725-44d8-b652-94613bd8137f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f95e0c579a7421cbe237f5715de337d', 'request_id': 'req-647f72c5-e725-44d8-b652-94613bd8137f'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:00.310073 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-7c5c1b6d-7c0a-4c19-b28f-17b91e78d739 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c488f6c9420498281252528d879a29b', 'request_id': 'req-7c5c1b6d-7c0a-4c19-b28f-17b91e78d739'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:00.328505 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-647f72c5-e725-44d8-b652-94613bd8137f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f95e0c579a7421cbe237f5715de337d', 'request_id': 'req-647f72c5-e725-44d8-b652-94613bd8137f'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:00.345439 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-7c5c1b6d-7c0a-4c19-b28f-17b91e78d739 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c488f6c9420498281252528d879a29b', 'request_id': 'req-7c5c1b6d-7c0a-4c19-b28f-17b91e78d739'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:00.368013 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-7c5c1b6d-7c0a-4c19-b28f-17b91e78d739 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c488f6c9420498281252528d879a29b', 'request_id': 'req-7c5c1b6d-7c0a-4c19-b28f-17b91e78d739'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:00.368163 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-647f72c5-e725-44d8-b652-94613bd8137f admin admin] The token cache is being invalidate because project 8f95e0c579a7421cbe237f5715de337d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:00.368496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-647f72c5-e725-44d8-b652-94613bd8137f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:00.370254 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-647f72c5-e725-44d8-b652-94613bd8137f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:00.375065 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 919/1859] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:24:00 2022] DELETE /identity/v3/projects/8f95e0c579a7421cbe237f5715de337d => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:00.379489 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f72c5-e725-44d8-b652-94613bd8137f admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:00.381762 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-647f72c5-e725-44d8-b652-94613bd8137f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:00.401679 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-7c5c1b6d-7c0a-4c19-b28f-17b91e78d739 admin admin] The token cache is being invalidate because project 1c488f6c9420498281252528d879a29b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:00.401890 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-7c5c1b6d-7c0a-4c19-b28f-17b91e78d739 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:00.406099 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-7c5c1b6d-7c0a-4c19-b28f-17b91e78d739 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:00.408508 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 941/1860] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:24:00 2022] DELETE /identity/v3/projects/1c488f6c9420498281252528d879a29b => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:00.414495 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c1b6d-7c0a-4c19-b28f-17b91e78d739 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:00.429107 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-7c5c1b6d-7c0a-4c19-b28f-17b91e78d739 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:00.517817 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f72c5-e725-44d8-b652-94613bd8137f admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:00.551620 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:00.597102 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c1b6d-7c0a-4c19-b28f-17b91e78d739 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:00.612066 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40613c4-39e8-4895-a20f-892fcdedccde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:00.615499 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40613c4-39e8-4895-a20f-892fcdedccde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:00.615797 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40613c4-39e8-4895-a20f-892fcdedccde admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:00.616070 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914002ef-d4c3-46b2-b12b-09b6843cf6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:00.616197 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40613c4-39e8-4895-a20f-892fcdedccde admin admin] PATH_INFO: `/v3/users/a6776c73da2f4b7a885af4fa99ba3d13` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:00.617670 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914002ef-d4c3-46b2-b12b-09b6843cf6f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:00.617930 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914002ef-d4c3-46b2-b12b-09b6843cf6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:00.618327 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914002ef-d4c3-46b2-b12b-09b6843cf6f3 admin admin] PATH_INFO: `/v3/users/e42cd4d5bc064da0bbb76e3b1f869c56` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:00.619676 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40613c4-39e8-4895-a20f-892fcdedccde admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6776c73da2f4b7a885af4fa99ba3d13)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:00.619939 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914002ef-d4c3-46b2-b12b-09b6843cf6f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e42cd4d5bc064da0bbb76e3b1f869c56)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:00.623170 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914002ef-d4c3-46b2-b12b-09b6843cf6f3 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:00.623976 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40613c4-39e8-4895-a20f-892fcdedccde admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:00.729320 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-914002ef-d4c3-46b2-b12b-09b6843cf6f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e42cd4d5bc064da0bbb76e3b1f869c56'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:00.758489 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-c40613c4-39e8-4895-a20f-892fcdedccde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6776c73da2f4b7a885af4fa99ba3d13'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:00.758489 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-c40613c4-39e8-4895-a20f-892fcdedccde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6776c73da2f4b7a885af4fa99ba3d13'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:00.759958 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-914002ef-d4c3-46b2-b12b-09b6843cf6f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e42cd4d5bc064da0bbb76e3b1f869c56'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:00.780324 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-c40613c4-39e8-4895-a20f-892fcdedccde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6776c73da2f4b7a885af4fa99ba3d13'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:00.782189 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-c40613c4-39e8-4895-a20f-892fcdedccde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6776c73da2f4b7a885af4fa99ba3d13'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:00.784492 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-914002ef-d4c3-46b2-b12b-09b6843cf6f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e42cd4d5bc064da0bbb76e3b1f869c56'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:00.801364 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-c40613c4-39e8-4895-a20f-892fcdedccde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6776c73da2f4b7a885af4fa99ba3d13'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:00.804385 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-914002ef-d4c3-46b2-b12b-09b6843cf6f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e42cd4d5bc064da0bbb76e3b1f869c56'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:00.813603 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-914002ef-d4c3-46b2-b12b-09b6843cf6f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e42cd4d5bc064da0bbb76e3b1f869c56'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:00.829004 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 920/1861] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:24:00 2022] DELETE /identity/v3/users/e42cd4d5bc064da0bbb76e3b1f869c56 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:24:00.845315 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 942/1862] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:24:00 2022] DELETE /identity/v3/users/a6776c73da2f4b7a885af4fa99ba3d13 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:01.889763 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914002ef-d4c3-46b2-b12b-09b6843cf6f3 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:01.893689 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-914002ef-d4c3-46b2-b12b-09b6843cf6f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:02.005291 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40613c4-39e8-4895-a20f-892fcdedccde admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:02.006747 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-c40613c4-39e8-4895-a20f-892fcdedccde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:02.033321 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914002ef-d4c3-46b2-b12b-09b6843cf6f3 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:02.138311 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:02.152551 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:02.167522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7eddc-7cb1-4d1c-877a-1283d5032fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:02.168481 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7eddc-7cb1-4d1c-877a-1283d5032fe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:02.168769 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7eddc-7cb1-4d1c-877a-1283d5032fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:02.169093 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7eddc-7cb1-4d1c-877a-1283d5032fe8 admin admin] PATH_INFO: `/v3/projects/2f3a2da744c648ffa7f50045955574be` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:02.170565 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb7eddc-7cb1-4d1c-877a-1283d5032fe8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f3a2da744c648ffa7f50045955574be)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:02.173248 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb7eddc-7cb1-4d1c-877a-1283d5032fe8 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:02.183474 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40613c4-39e8-4895-a20f-892fcdedccde admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:02.188874 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1143c1da-cca0-4cb8-8618-ed73bdf1e100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:02.189702 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143c1da-cca0-4cb8-8618-ed73bdf1e100 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:02.189981 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143c1da-cca0-4cb8-8618-ed73bdf1e100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:02.190255 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1143c1da-cca0-4cb8-8618-ed73bdf1e100 admin admin] PATH_INFO: `/v3/projects/9115fc4f8d5b4718a74c1e1497e26f41` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:02.191859 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1143c1da-cca0-4cb8-8618-ed73bdf1e100 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9115fc4f8d5b4718a74c1e1497e26f41)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:02.195428 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1143c1da-cca0-4cb8-8618-ed73bdf1e100 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:02.243613 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-beb7eddc-7cb1-4d1c-877a-1283d5032fe8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f3a2da744c648ffa7f50045955574be', 'request_id': 'req-beb7eddc-7cb1-4d1c-877a-1283d5032fe8'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:02.252514 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-beb7eddc-7cb1-4d1c-877a-1283d5032fe8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f3a2da744c648ffa7f50045955574be', 'request_id': 'req-beb7eddc-7cb1-4d1c-877a-1283d5032fe8'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:02.263291 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-beb7eddc-7cb1-4d1c-877a-1283d5032fe8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f3a2da744c648ffa7f50045955574be', 'request_id': 'req-beb7eddc-7cb1-4d1c-877a-1283d5032fe8'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:02.296399 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-1143c1da-cca0-4cb8-8618-ed73bdf1e100 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9115fc4f8d5b4718a74c1e1497e26f41', 'request_id': 'req-1143c1da-cca0-4cb8-8618-ed73bdf1e100'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:02.300331 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-beb7eddc-7cb1-4d1c-877a-1283d5032fe8 admin admin] The token cache is being invalidate because project 2f3a2da744c648ffa7f50045955574be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:02.302016 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-beb7eddc-7cb1-4d1c-877a-1283d5032fe8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:02.302712 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-beb7eddc-7cb1-4d1c-877a-1283d5032fe8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:02.304760 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 921/1863] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:24:01 2022] DELETE /identity/v3/projects/2f3a2da744c648ffa7f50045955574be => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:02.309448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-1143c1da-cca0-4cb8-8618-ed73bdf1e100 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9115fc4f8d5b4718a74c1e1497e26f41', 'request_id': 'req-1143c1da-cca0-4cb8-8618-ed73bdf1e100'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:02.310366 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7eddc-7cb1-4d1c-877a-1283d5032fe8 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:02.311936 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-beb7eddc-7cb1-4d1c-877a-1283d5032fe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:02.328782 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-1143c1da-cca0-4cb8-8618-ed73bdf1e100 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9115fc4f8d5b4718a74c1e1497e26f41', 'request_id': 'req-1143c1da-cca0-4cb8-8618-ed73bdf1e100'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:02.348135 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-1143c1da-cca0-4cb8-8618-ed73bdf1e100 admin admin] The token cache is being invalidate because project 9115fc4f8d5b4718a74c1e1497e26f41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:02.348671 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-1143c1da-cca0-4cb8-8618-ed73bdf1e100 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:02.349643 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-1143c1da-cca0-4cb8-8618-ed73bdf1e100 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:02.357278 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 943/1864] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:24:02 2022] DELETE /identity/v3/projects/9115fc4f8d5b4718a74c1e1497e26f41 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:02.359948 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ff6b0-bd1c-4784-bff5-4f5f2280d75b None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:02.360239 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ff6b0-bd1c-4784-bff5-4f5f2280d75b None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:02.360707 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ff6b0-bd1c-4784-bff5-4f5f2280d75b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:02.416192 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7eddc-7cb1-4d1c-877a-1283d5032fe8 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:02.418109 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-2b4ff6b0-bd1c-4784-bff5-4f5f2280d75b None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:02.419480 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-2b4ff6b0-bd1c-4784-bff5-4f5f2280d75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:02.511413 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058591a6-91f5-4c17-a3db-0401b302036b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:02.513357 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058591a6-91f5-4c17-a3db-0401b302036b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:02.513768 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058591a6-91f5-4c17-a3db-0401b302036b admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:02.514119 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058591a6-91f5-4c17-a3db-0401b302036b admin admin] PATH_INFO: `/v3/users/158489cfd87e4d2cb82d07e45b641aed` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:02.516749 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058591a6-91f5-4c17-a3db-0401b302036b admin admin] RBAC: Authorizing `identity:delete_user(user_id=158489cfd87e4d2cb82d07e45b641aed)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:02.518846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058591a6-91f5-4c17-a3db-0401b302036b admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:02.568730 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 944/1865] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:24:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:02.578619 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc63eb-40e2-4487-8bd1-757de5b260fa None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:02.578619 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc63eb-40e2-4487-8bd1-757de5b260fa None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:02.578619 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc63eb-40e2-4487-8bd1-757de5b260fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:02.581461 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:02.630664 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-058591a6-91f5-4c17-a3db-0401b302036b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '158489cfd87e4d2cb82d07e45b641aed'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:02.644561 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-25fc63eb-40e2-4487-8bd1-757de5b260fa None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:02.646840 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-25fc63eb-40e2-4487-8bd1-757de5b260fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:02.662539 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 945/1866] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:24:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:02.663728 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-058591a6-91f5-4c17-a3db-0401b302036b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '158489cfd87e4d2cb82d07e45b641aed'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:02.667560 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fc63eb-40e2-4487-8bd1-757de5b260fa None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:02.681900 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fc63eb-40e2-4487-8bd1-757de5b260fa None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:02.685965 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-058591a6-91f5-4c17-a3db-0401b302036b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '158489cfd87e4d2cb82d07e45b641aed'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:02.689312 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc31192-ec6a-40d7-a4b7-873e0260a038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:02.690017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc31192-ec6a-40d7-a4b7-873e0260a038 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:02.690831 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc31192-ec6a-40d7-a4b7-873e0260a038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:02.691167 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc31192-ec6a-40d7-a4b7-873e0260a038 admin admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:02.692219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc31192-ec6a-40d7-a4b7-873e0260a038 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:24:02.692544 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc31192-ec6a-40d7-a4b7-873e0260a038 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:02.694770 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc31192-ec6a-40d7-a4b7-873e0260a038 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:02.696229 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-058591a6-91f5-4c17-a3db-0401b302036b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '158489cfd87e4d2cb82d07e45b641aed'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:02.697312 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-058591a6-91f5-4c17-a3db-0401b302036b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '158489cfd87e4d2cb82d07e45b641aed'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:02.713088 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 922/1867] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:24:02 2022] DELETE /identity/v3/users/158489cfd87e4d2cb82d07e45b641aed => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:02.733444 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 946/1868] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:24:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:24:02.738335 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058591a6-91f5-4c17-a3db-0401b302036b admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:02.740429 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-058591a6-91f5-4c17-a3db-0401b302036b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:02.852303 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058591a6-91f5-4c17-a3db-0401b302036b admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:02.857842 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c32e0b3-f7ed-444c-ba7d-7edc420fe8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:02.858524 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32e0b3-f7ed-444c-ba7d-7edc420fe8ef admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:02.858807 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32e0b3-f7ed-444c-ba7d-7edc420fe8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:02.859081 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c32e0b3-f7ed-444c-ba7d-7edc420fe8ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:02.860148 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c32e0b3-f7ed-444c-ba7d-7edc420fe8ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:02.861891 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c32e0b3-f7ed-444c-ba7d-7edc420fe8ef admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:02.899737 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 923/1869] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:24:02 2022] POST /identity/v3/projects => generated 407 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:02.902703 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc31192-ec6a-40d7-a4b7-873e0260a038 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:02.928897 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc31192-ec6a-40d7-a4b7-873e0260a038 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:02.988625 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:03.003552 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98808d5f-28db-4ebd-b7e1-0995c3084534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:03.004344 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98808d5f-28db-4ebd-b7e1-0995c3084534 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:03.004682 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98808d5f-28db-4ebd-b7e1-0995c3084534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:03.005080 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98808d5f-28db-4ebd-b7e1-0995c3084534 admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:03.005854 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98808d5f-28db-4ebd-b7e1-0995c3084534 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:03.008333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98808d5f-28db-4ebd-b7e1-0995c3084534 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:03.032054 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 947/1870] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:24:02 2022] POST /identity/v3/users => generated 304 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:24:03.036526 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c32e0b3-f7ed-444c-ba7d-7edc420fe8ef admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:03.047513 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c32e0b3-f7ed-444c-ba7d-7edc420fe8ef admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:03.053506 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4640289-eb27-4e0c-8761-e78006cf0fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:03.054211 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4640289-eb27-4e0c-8761-e78006cf0fbf admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:03.054930 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4640289-eb27-4e0c-8761-e78006cf0fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:03.054930 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4640289-eb27-4e0c-8761-e78006cf0fbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:03.056579 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4640289-eb27-4e0c-8761-e78006cf0fbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:03.057650 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4640289-eb27-4e0c-8761-e78006cf0fbf admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:03.080791 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98808d5f-28db-4ebd-b7e1-0995c3084534 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:03.081340 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 924/1871] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:03.085570 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4640289-eb27-4e0c-8761-e78006cf0fbf admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:03.086025 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-98808d5f-28db-4ebd-b7e1-0995c3084534 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:03.098341 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4640289-eb27-4e0c-8761-e78006cf0fbf admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:03.100991 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98808d5f-28db-4ebd-b7e1-0995c3084534 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:03.106514 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb681e92-0855-407e-897f-48de43233271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:03.107436 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb681e92-0855-407e-897f-48de43233271 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:03.107701 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb681e92-0855-407e-897f-48de43233271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:03.107975 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb681e92-0855-407e-897f-48de43233271 admin admin] PATH_INFO: `/v3/projects/350c2f8f5adb4b44992dd12b32d5d11e` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:03.109190 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2a5f0e-c22b-4a28-bcf7-77fd004ec4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:03.109339 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb681e92-0855-407e-897f-48de43233271 admin admin] RBAC: Authorizing `identity:delete_project(project_id=350c2f8f5adb4b44992dd12b32d5d11e)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:03.110028 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a5f0e-c22b-4a28-bcf7-77fd004ec4b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:03.110405 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a5f0e-c22b-4a28-bcf7-77fd004ec4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:03.110790 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2a5f0e-c22b-4a28-bcf7-77fd004ec4b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:03.111236 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb681e92-0855-407e-897f-48de43233271 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:03.111655 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2a5f0e-c22b-4a28-bcf7-77fd004ec4b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:03.113557 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2a5f0e-c22b-4a28-bcf7-77fd004ec4b7 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:03.125158 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 925/1872] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:03.129295 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2a5f0e-c22b-4a28-bcf7-77fd004ec4b7 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:03.140987 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2a5f0e-c22b-4a28-bcf7-77fd004ec4b7 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:03.145904 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b624b-0b66-4721-b6cb-334704727c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:03.146835 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b624b-0b66-4721-b6cb-334704727c9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:03.147117 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b624b-0b66-4721-b6cb-334704727c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:03.147408 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b624b-0b66-4721-b6cb-334704727c9d admin admin] PATH_INFO: `/v3/projects/fe675b007ea14d1d8e3adf534a868eeb/users/802bfe2803b347f09da2334cc699ffea/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:03.163886 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b624b-0b66-4721-b6cb-334704727c9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe675b007ea14d1d8e3adf534a868eeb, user_id=802bfe2803b347f09da2334cc699ffea, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:03.168094 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b624b-0b66-4721-b6cb-334704727c9d admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:03.169937 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-eb681e92-0855-407e-897f-48de43233271 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '350c2f8f5adb4b44992dd12b32d5d11e', 'request_id': 'req-eb681e92-0855-407e-897f-48de43233271'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:03.177073 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-eb681e92-0855-407e-897f-48de43233271 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '350c2f8f5adb4b44992dd12b32d5d11e', 'request_id': 'req-eb681e92-0855-407e-897f-48de43233271'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:03.182691 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 926/1873] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:24:03 2022] PUT /identity/v3/projects/fe675b007ea14d1d8e3adf534a868eeb/users/802bfe2803b347f09da2334cc699ffea/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:24:03.187938 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-eb681e92-0855-407e-897f-48de43233271 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '350c2f8f5adb4b44992dd12b32d5d11e', 'request_id': 'req-eb681e92-0855-407e-897f-48de43233271'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:03.194850 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a21e96-7948-4b41-83ba-f90ffb1eff05 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:03.195287 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a21e96-7948-4b41-83ba-f90ffb1eff05 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:03.195570 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a21e96-7948-4b41-83ba-f90ffb1eff05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:03.199733 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-eb681e92-0855-407e-897f-48de43233271 admin admin] The token cache is being invalidate because project 350c2f8f5adb4b44992dd12b32d5d11e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:03.200112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-eb681e92-0855-407e-897f-48de43233271 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:03.200829 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-eb681e92-0855-407e-897f-48de43233271 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:03.202575 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 948/1874] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:24:03 2022] DELETE /identity/v3/projects/350c2f8f5adb4b44992dd12b32d5d11e => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:03.208789 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb0dc4-b5d7-438e-8321-ab2a9c7320f6 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:03.209118 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb0dc4-b5d7-438e-8321-ab2a9c7320f6 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:03.209399 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb0dc4-b5d7-438e-8321-ab2a9c7320f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:03.232594 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-4fbb0dc4-b5d7-438e-8321-ab2a9c7320f6 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:03.234006 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-18a21e96-7948-4b41-83ba-f90ffb1eff05 None None] MFA Rules not processed for user `802bfe2803b347f09da2334cc699ffea`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:03.234781 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-4fbb0dc4-b5d7-438e-8321-ab2a9c7320f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:03.234899 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-18a21e96-7948-4b41-83ba-f90ffb1eff05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:03.278789 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:03.324180 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 949/1876] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:24:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:24:03.324418 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 927/1876] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:24:03 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:24:03.328422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb0dc4-b5d7-438e-8321-ab2a9c7320f6 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:03.329946 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-4fbb0dc4-b5d7-438e-8321-ab2a9c7320f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:03.331986 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab6c84-48ce-464f-9ed3-37a4eb96f9ab None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:03.332431 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab6c84-48ce-464f-9ed3-37a4eb96f9ab None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:03.332708 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab6c84-48ce-464f-9ed3-37a4eb96f9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:03.348925 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb0dc4-b5d7-438e-8321-ab2a9c7320f6 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:03.354469 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde7955-5503-4345-9a74-0a3b2ac618e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:03.355304 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde7955-5503-4345-9a74-0a3b2ac618e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:03.355656 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde7955-5503-4345-9a74-0a3b2ac618e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:03.355935 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde7955-5503-4345-9a74-0a3b2ac618e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:03.356618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-66ab6c84-48ce-464f-9ed3-37a4eb96f9ab None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:03.356896 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cde7955-5503-4345-9a74-0a3b2ac618e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:03.357880 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-66ab6c84-48ce-464f-9ed3-37a4eb96f9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:03.358896 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cde7955-5503-4345-9a74-0a3b2ac618e0 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:03.371319 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 928/1877] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:24:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:24:03.375431 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab6c84-48ce-464f-9ed3-37a4eb96f9ab None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:03.389527 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab6c84-48ce-464f-9ed3-37a4eb96f9ab None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:03.399143 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe8ede-b213-44aa-aa1b-e2964abd5b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:03.400130 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 950/1878] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:24:03 2022] POST /identity/v3/projects => generated 407 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:24:03.400682 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde7955-5503-4345-9a74-0a3b2ac618e0 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:03.400992 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe8ede-b213-44aa-aa1b-e2964abd5b0c admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:03.401510 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe8ede-b213-44aa-aa1b-e2964abd5b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:03.401793 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe8ede-b213-44aa-aa1b-e2964abd5b0c admin admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:03.402615 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe8ede-b213-44aa-aa1b-e2964abd5b0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:24:03.403844 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe8ede-b213-44aa-aa1b-e2964abd5b0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:03.405892 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe8ede-b213-44aa-aa1b-e2964abd5b0c admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:03.423894 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 929/1879] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:24:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:24:03.428167 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe8ede-b213-44aa-aa1b-e2964abd5b0c admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:03.439330 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe8ede-b213-44aa-aa1b-e2964abd5b0c admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:03.458802 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:03.488833 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde7955-5503-4345-9a74-0a3b2ac618e0 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:03.495527 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3609b-f0fc-4760-9edb-5698a54d7e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:03.496415 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3609b-f0fc-4760-9edb-5698a54d7e0c admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:03.496702 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3609b-f0fc-4760-9edb-5698a54d7e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:03.498316 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3609b-f0fc-4760-9edb-5698a54d7e0c admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:03.498316 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3609b-f0fc-4760-9edb-5698a54d7e0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:03.499679 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3609b-f0fc-4760-9edb-5698a54d7e0c admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:03.519139 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96730c4b-68c1-4821-8b04-0810cb508e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:03.520115 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96730c4b-68c1-4821-8b04-0810cb508e99 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:03.520489 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96730c4b-68c1-4821-8b04-0810cb508e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:03.520991 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96730c4b-68c1-4821-8b04-0810cb508e99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:03.521921 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96730c4b-68c1-4821-8b04-0810cb508e99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:03.524288 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96730c4b-68c1-4821-8b04-0810cb508e99 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:03.525342 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 951/1880] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:24:03 2022] POST /identity/v3/users => generated 310 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:03.529503 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3609b-f0fc-4760-9edb-5698a54d7e0c admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:03.541787 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3609b-f0fc-4760-9edb-5698a54d7e0c admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:03.547313 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24211b-bde2-492a-a40c-08f1e48fad24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:03.548199 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24211b-bde2-492a-a40c-08f1e48fad24 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:03.548398 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24211b-bde2-492a-a40c-08f1e48fad24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:03.548719 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24211b-bde2-492a-a40c-08f1e48fad24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:03.549648 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae24211b-bde2-492a-a40c-08f1e48fad24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:03.551459 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae24211b-bde2-492a-a40c-08f1e48fad24 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:03.566663 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 952/1881] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:24:03.570700 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24211b-bde2-492a-a40c-08f1e48fad24 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:03.577500 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 930/1882] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:24:03 2022] POST /identity/v3/projects => generated 379 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:24:03.583180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96730c4b-68c1-4821-8b04-0810cb508e99 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:03.583665 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae24211b-bde2-492a-a40c-08f1e48fad24 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:03.699024 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96730c4b-68c1-4821-8b04-0810cb508e99 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:03.711093 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def9214-6b5f-4d43-bf11-44cc4f5c33ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:03.713235 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def9214-6b5f-4d43-bf11-44cc4f5c33ca admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:03.714389 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def9214-6b5f-4d43-bf11-44cc4f5c33ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:03.714864 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def9214-6b5f-4d43-bf11-44cc4f5c33ca admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:03.718373 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9def9214-6b5f-4d43-bf11-44cc4f5c33ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:03.722682 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25efc08-42ed-4e81-ae3d-9e0ce1953787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:03.722854 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9def9214-6b5f-4d43-bf11-44cc4f5c33ca admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:03.724117 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25efc08-42ed-4e81-ae3d-9e0ce1953787 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:03.724495 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25efc08-42ed-4e81-ae3d-9e0ce1953787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:03.725065 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25efc08-42ed-4e81-ae3d-9e0ce1953787 admin admin] PATH_INFO: `/v3/projects/47319ede4d6047259553de024723a3ab/users/ea821f6e114b473f8f7578273f5454f3/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:03.758230 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25efc08-42ed-4e81-ae3d-9e0ce1953787 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47319ede4d6047259553de024723a3ab, user_id=ea821f6e114b473f8f7578273f5454f3, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:03.768832 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25efc08-42ed-4e81-ae3d-9e0ce1953787 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:03.788830 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 931/1883] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:24:03 2022] POST /identity/v3/users => generated 290 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:03.792144 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 953/1884] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:24:03 2022] PUT /identity/v3/projects/47319ede4d6047259553de024723a3ab/users/ea821f6e114b473f8f7578273f5454f3/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:24:03.792334 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def9214-6b5f-4d43-bf11-44cc4f5c33ca admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:03.793645 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25efc08-42ed-4e81-ae3d-9e0ce1953787 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:03.818399 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def9214-6b5f-4d43-bf11-44cc4f5c33ca admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:03.856063 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:03.908348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3077152-82f3-4515-8a04-5af2e231804d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:03.908941 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3077152-82f3-4515-8a04-5af2e231804d admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:03.908941 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3077152-82f3-4515-8a04-5af2e231804d admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:03.909304 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3077152-82f3-4515-8a04-5af2e231804d admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:03.909990 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3077152-82f3-4515-8a04-5af2e231804d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:03.911589 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3077152-82f3-4515-8a04-5af2e231804d admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:03.912144 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25efc08-42ed-4e81-ae3d-9e0ce1953787 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:03.918383 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a3a5b-1861-4cb8-b4ef-ee1c7abbc774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:03.919627 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a3a5b-1861-4cb8-b4ef-ee1c7abbc774 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:03.919850 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a3a5b-1861-4cb8-b4ef-ee1c7abbc774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:03.920128 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a3a5b-1861-4cb8-b4ef-ee1c7abbc774 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:03.921464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0a3a5b-1861-4cb8-b4ef-ee1c7abbc774 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:03.923708 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0a3a5b-1861-4cb8-b4ef-ee1c7abbc774 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:03.938501 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 932/1885] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:03.939500 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3077152-82f3-4515-8a04-5af2e231804d admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:03.942682 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 954/1886] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:03.946919 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a3a5b-1861-4cb8-b4ef-ee1c7abbc774 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:03.955622 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3077152-82f3-4515-8a04-5af2e231804d admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:03.960458 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add0f16b-cfae-4cdf-af38-a4b2bab60d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:03.961336 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0f16b-cfae-4cdf-af38-a4b2bab60d26 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:03.961617 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0f16b-cfae-4cdf-af38-a4b2bab60d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:03.961882 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0f16b-cfae-4cdf-af38-a4b2bab60d26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:03.963586 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add0f16b-cfae-4cdf-af38-a4b2bab60d26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:03.965176 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add0f16b-cfae-4cdf-af38-a4b2bab60d26 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:03.966424 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a3a5b-1861-4cb8-b4ef-ee1c7abbc774 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:03.973137 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158c57e-8f0a-4084-b2e9-860910df561c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:03.974883 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158c57e-8f0a-4084-b2e9-860910df561c admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:03.975348 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158c57e-8f0a-4084-b2e9-860910df561c admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:03.975827 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158c57e-8f0a-4084-b2e9-860910df561c admin admin] PATH_INFO: `/v3/projects/47319ede4d6047259553de024723a3ab/users/ea821f6e114b473f8f7578273f5454f3/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:03.984204 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 933/1887] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:03.984865 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158c57e-8f0a-4084-b2e9-860910df561c admin admin] RBAC: Authorizing `identity:create_grant(project_id=47319ede4d6047259553de024723a3ab, user_id=ea821f6e114b473f8f7578273f5454f3, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:03.985567 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add0f16b-cfae-4cdf-af38-a4b2bab60d26 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:03.987913 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158c57e-8f0a-4084-b2e9-860910df561c admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:03.996614 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add0f16b-cfae-4cdf-af38-a4b2bab60d26 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:04.007489 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 955/1888] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:24:03 2022] PUT /identity/v3/projects/47319ede4d6047259553de024723a3ab/users/ea821f6e114b473f8f7578273f5454f3/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:04.025980 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ace951-7693-4d8d-856b-0746413e5460 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:04.025980 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ace951-7693-4d8d-856b-0746413e5460 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:04.025980 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ace951-7693-4d8d-856b-0746413e5460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:04.070592 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-61ace951-7693-4d8d-856b-0746413e5460 None None] MFA Rules not processed for user `ea821f6e114b473f8f7578273f5454f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:04.071899 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-61ace951-7693-4d8d-856b-0746413e5460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:04.086375 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:04.125181 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760c1a2-bcda-424f-a01e-4be42178cab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:04.126198 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760c1a2-bcda-424f-a01e-4be42178cab6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:04.126481 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760c1a2-bcda-424f-a01e-4be42178cab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:04.126757 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760c1a2-bcda-424f-a01e-4be42178cab6 admin admin] PATH_INFO: `/v3/projects/d5bef1b49d424299864523887eb72b3c/users/01be6ea7136e45aead352146193f05ed/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:04.153917 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760c1a2-bcda-424f-a01e-4be42178cab6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5bef1b49d424299864523887eb72b3c, user_id=01be6ea7136e45aead352146193f05ed, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:04.156879 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760c1a2-bcda-424f-a01e-4be42178cab6 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:04.183869 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 956/1889] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:24:04 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:04.194080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 934/1890] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:24:03 2022] PUT /identity/v3/projects/d5bef1b49d424299864523887eb72b3c/users/01be6ea7136e45aead352146193f05ed/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:04.198165 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ace951-7693-4d8d-856b-0746413e5460 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:04.199009 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f230a-df8b-4001-8f2e-bb9566fe0de7 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:04.199298 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f230a-df8b-4001-8f2e-bb9566fe0de7 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:04.199577 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f230a-df8b-4001-8f2e-bb9566fe0de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:04.200109 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-61ace951-7693-4d8d-856b-0746413e5460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:04.221932 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-239f230a-df8b-4001-8f2e-bb9566fe0de7 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:04.223830 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-239f230a-df8b-4001-8f2e-bb9566fe0de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:04.275515 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ace951-7693-4d8d-856b-0746413e5460 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:04.307152 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 935/1891] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:24:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:24:04.317575 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239f230a-df8b-4001-8f2e-bb9566fe0de7 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:04.319011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-239f230a-df8b-4001-8f2e-bb9566fe0de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:04.340974 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590f6d6d-05cb-4ddf-91b2-84778ae5aa3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:04.341696 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f6d6d-05cb-4ddf-91b2-84778ae5aa3c service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:04.341973 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f6d6d-05cb-4ddf-91b2-84778ae5aa3c service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:04.342241 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f6d6d-05cb-4ddf-91b2-84778ae5aa3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:04.369570 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:04.386397 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239f230a-df8b-4001-8f2e-bb9566fe0de7 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:04.389985 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a814ed68-d5d8-4089-b147-4414464f8d89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:04.390981 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814ed68-d5d8-4089-b147-4414464f8d89 service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:04.391376 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814ed68-d5d8-4089-b147-4414464f8d89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:04.391676 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814ed68-d5d8-4089-b147-4414464f8d89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:04.402015 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a814ed68-d5d8-4089-b147-4414464f8d89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:04.403514 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a814ed68-d5d8-4089-b147-4414464f8d89 service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:04.407841 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f6d6d-05cb-4ddf-91b2-84778ae5aa3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:04.409345 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590f6d6d-05cb-4ddf-91b2-84778ae5aa3c service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:04.413376 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 936/1892] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:24:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:24:04.420033 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 957/1893] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:24:04 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:04.641708 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a814ed68-d5d8-4089-b147-4414464f8d89 service neutron] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:04.655184 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a814ed68-d5d8-4089-b147-4414464f8d89 service neutron] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:04.666345 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010cde50-cc8d-4418-9f3a-19c33c4e7c90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:04.667034 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010cde50-cc8d-4418-9f3a-19c33c4e7c90 service cinder] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:04.667319 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010cde50-cc8d-4418-9f3a-19c33c4e7c90 service cinder] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:04.667593 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010cde50-cc8d-4418-9f3a-19c33c4e7c90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:04.740331 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010cde50-cc8d-4418-9f3a-19c33c4e7c90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:04.742125 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010cde50-cc8d-4418-9f3a-19c33c4e7c90 service cinder] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:04.753362 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 937/1894] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:24:04 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:10.176586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b4914-0b3c-4b5f-b98c-b6c6d39264c2 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:10.177715 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b4914-0b3c-4b5f-b98c-b6c6d39264c2 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:10.178166 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b4914-0b3c-4b5f-b98c-b6c6d39264c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:10.212621 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:10.214339 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-388b4914-0b3c-4b5f-b98c-b6c6d39264c2 None None] MFA Rules not processed for user `01be6ea7136e45aead352146193f05ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:10.215997 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-388b4914-0b3c-4b5f-b98c-b6c6d39264c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:10.296089 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 958/1895] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:24:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:10.297030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010cde50-cc8d-4418-9f3a-19c33c4e7c90 service cinder] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:10.308476 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010cde50-cc8d-4418-9f3a-19c33c4e7c90 service cinder] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:10.313050 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a08997c-ad99-4de9-989c-b88d05e9efe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:10.313726 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08997c-ad99-4de9-989c-b88d05e9efe4 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:10.314007 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08997c-ad99-4de9-989c-b88d05e9efe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:10.314288 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08997c-ad99-4de9-989c-b88d05e9efe4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:10.314981 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a08997c-ad99-4de9-989c-b88d05e9efe4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:10.316991 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a08997c-ad99-4de9-989c-b88d05e9efe4 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:10.355582 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 938/1896] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:24:10 2022] POST /identity/v3/projects => generated 377 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:24:10.359879 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b4914-0b3c-4b5f-b98c-b6c6d39264c2 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:10.387059 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b4914-0b3c-4b5f-b98c-b6c6d39264c2 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:10.477679 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3527653a-a1f5-44e2-8cad-637092ab12cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:10.478490 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527653a-a1f5-44e2-8cad-637092ab12cb admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:10.478863 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527653a-a1f5-44e2-8cad-637092ab12cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:10.479164 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527653a-a1f5-44e2-8cad-637092ab12cb admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:10.481139 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3527653a-a1f5-44e2-8cad-637092ab12cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:10.484874 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3527653a-a1f5-44e2-8cad-637092ab12cb admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:10.514762 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 959/1897] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:24:10 2022] POST /identity/v3/users => generated 295 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:24:10.516167 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a08997c-ad99-4de9-989c-b88d05e9efe4 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:10.530891 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a08997c-ad99-4de9-989c-b88d05e9efe4 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:10.536763 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd166222-bb07-4cba-9845-1be0238f3904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:10.537660 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd166222-bb07-4cba-9845-1be0238f3904 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:10.538012 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd166222-bb07-4cba-9845-1be0238f3904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:10.538581 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd166222-bb07-4cba-9845-1be0238f3904 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:10.539696 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd166222-bb07-4cba-9845-1be0238f3904 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:10.542340 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd166222-bb07-4cba-9845-1be0238f3904 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:10.560344 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 939/1898] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:10.561661 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3527653a-a1f5-44e2-8cad-637092ab12cb admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:10.575026 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3527653a-a1f5-44e2-8cad-637092ab12cb admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:10.578346 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9817de1-77c8-4181-87e3-6cedae47622f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:10.579448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9817de1-77c8-4181-87e3-6cedae47622f admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:10.579731 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9817de1-77c8-4181-87e3-6cedae47622f admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:10.580004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9817de1-77c8-4181-87e3-6cedae47622f admin admin] PATH_INFO: `/v3/projects/1021e1aa0d4143d8a6513d484f48f601/users/627f0217ac8344318fb58db6dc918649/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:10.602414 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9817de1-77c8-4181-87e3-6cedae47622f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1021e1aa0d4143d8a6513d484f48f601, user_id=627f0217ac8344318fb58db6dc918649, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:10.606349 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9817de1-77c8-4181-87e3-6cedae47622f admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:10.640725 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 960/1899] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:24:10 2022] PUT /identity/v3/projects/1021e1aa0d4143d8a6513d484f48f601/users/627f0217ac8344318fb58db6dc918649/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:10.643112 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd166222-bb07-4cba-9845-1be0238f3904 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:10.654501 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd166222-bb07-4cba-9845-1be0238f3904 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:10.758004 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17055d-4cb6-476f-b51f-74f63a42ceed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:10.759001 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17055d-4cb6-476f-b51f-74f63a42ceed admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:10.759278 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17055d-4cb6-476f-b51f-74f63a42ceed admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:10.759548 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17055d-4cb6-476f-b51f-74f63a42ceed admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:10.760278 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f17055d-4cb6-476f-b51f-74f63a42ceed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:10.761914 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f17055d-4cb6-476f-b51f-74f63a42ceed admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:10.776914 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 940/1900] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:24:10.778384 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9817de1-77c8-4181-87e3-6cedae47622f admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:10.790516 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9817de1-77c8-4181-87e3-6cedae47622f admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:10.798501 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ee813-0384-43b1-b26f-861b06dbb4aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:10.799423 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ee813-0384-43b1-b26f-861b06dbb4aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:10.799702 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ee813-0384-43b1-b26f-861b06dbb4aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:10.799976 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ee813-0384-43b1-b26f-861b06dbb4aa admin admin] PATH_INFO: `/v3/projects/1021e1aa0d4143d8a6513d484f48f601/users/627f0217ac8344318fb58db6dc918649/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:10.804519 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7ee813-0384-43b1-b26f-861b06dbb4aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=1021e1aa0d4143d8a6513d484f48f601, user_id=627f0217ac8344318fb58db6dc918649, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:10.807377 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7ee813-0384-43b1-b26f-861b06dbb4aa admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:10.823501 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 961/1901] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:24:10 2022] PUT /identity/v3/projects/1021e1aa0d4143d8a6513d484f48f601/users/627f0217ac8344318fb58db6dc918649/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:14.156037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17055d-4cb6-476f-b51f-74f63a42ceed admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:14.168509 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17055d-4cb6-476f-b51f-74f63a42ceed admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:14.325396 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:14.360552 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd733b0-3965-4e36-9d86-500fb2bec7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:14.361516 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd733b0-3965-4e36-9d86-500fb2bec7a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:14.361789 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd733b0-3965-4e36-9d86-500fb2bec7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:14.362058 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd733b0-3965-4e36-9d86-500fb2bec7a3 admin admin] PATH_INFO: `/v3/users/802bfe2803b347f09da2334cc699ffea` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:14.368522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd733b0-3965-4e36-9d86-500fb2bec7a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=802bfe2803b347f09da2334cc699ffea)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:14.370585 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd733b0-3965-4e36-9d86-500fb2bec7a3 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:14.499168 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-8bd733b0-3965-4e36-9d86-500fb2bec7a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '802bfe2803b347f09da2334cc699ffea'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:14.526809 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-8bd733b0-3965-4e36-9d86-500fb2bec7a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '802bfe2803b347f09da2334cc699ffea'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:14.542190 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-8bd733b0-3965-4e36-9d86-500fb2bec7a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '802bfe2803b347f09da2334cc699ffea'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:14.555892 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-8bd733b0-3965-4e36-9d86-500fb2bec7a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '802bfe2803b347f09da2334cc699ffea'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:14.557554 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-8bd733b0-3965-4e36-9d86-500fb2bec7a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '802bfe2803b347f09da2334cc699ffea'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:14.568401 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 941/1902] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:24:14 2022] DELETE /identity/v3/users/802bfe2803b347f09da2334cc699ffea => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:14.573996 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77236c9c-e8d5-4e74-a548-a8286d9af44d None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:14.574612 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77236c9c-e8d5-4e74-a548-a8286d9af44d None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:14.574717 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77236c9c-e8d5-4e74-a548-a8286d9af44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:14.622260 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-77236c9c-e8d5-4e74-a548-a8286d9af44d None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:14.623580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-77236c9c-e8d5-4e74-a548-a8286d9af44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:14.786548 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:14.812153 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 962/1903] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:24:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:14.830437 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd733b0-3965-4e36-9d86-500fb2bec7a3 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:14.832510 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-8bd733b0-3965-4e36-9d86-500fb2bec7a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:15.007480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd733b0-3965-4e36-9d86-500fb2bec7a3 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:15.016448 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0153df6-72d1-4b05-8a8f-e9ecf7c1528b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:15.019519 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0153df6-72d1-4b05-8a8f-e9ecf7c1528b service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:15.019912 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0153df6-72d1-4b05-8a8f-e9ecf7c1528b service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:15.020285 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0153df6-72d1-4b05-8a8f-e9ecf7c1528b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:15.051808 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0153df6-72d1-4b05-8a8f-e9ecf7c1528b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:15.055240 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0153df6-72d1-4b05-8a8f-e9ecf7c1528b service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:15.075755 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 942/1904] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:24:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:16.063073 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77236c9c-e8d5-4e74-a548-a8286d9af44d None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:16.065131 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-77236c9c-e8d5-4e74-a548-a8286d9af44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:16.092918 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77236c9c-e8d5-4e74-a548-a8286d9af44d None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:16.102134 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad30db-4f44-458c-ab2a-f9dc8b71748d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:16.104452 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad30db-4f44-458c-ab2a-f9dc8b71748d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:16.105387 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad30db-4f44-458c-ab2a-f9dc8b71748d admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:16.105841 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad30db-4f44-458c-ab2a-f9dc8b71748d admin admin] PATH_INFO: `/v3/projects/fe675b007ea14d1d8e3adf534a868eeb` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:16.109362 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad30db-4f44-458c-ab2a-f9dc8b71748d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe675b007ea14d1d8e3adf534a868eeb)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:16.113433 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad30db-4f44-458c-ab2a-f9dc8b71748d admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:16.202922 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-edad30db-4f44-458c-ab2a-f9dc8b71748d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe675b007ea14d1d8e3adf534a868eeb', 'request_id': 'req-edad30db-4f44-458c-ab2a-f9dc8b71748d'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:16.214966 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-edad30db-4f44-458c-ab2a-f9dc8b71748d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe675b007ea14d1d8e3adf534a868eeb', 'request_id': 'req-edad30db-4f44-458c-ab2a-f9dc8b71748d'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:16.230976 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-edad30db-4f44-458c-ab2a-f9dc8b71748d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe675b007ea14d1d8e3adf534a868eeb', 'request_id': 'req-edad30db-4f44-458c-ab2a-f9dc8b71748d'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:16.254350 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-edad30db-4f44-458c-ab2a-f9dc8b71748d admin admin] The token cache is being invalidate because project fe675b007ea14d1d8e3adf534a868eeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:16.259125 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-edad30db-4f44-458c-ab2a-f9dc8b71748d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:16.260444 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-edad30db-4f44-458c-ab2a-f9dc8b71748d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:16.262714 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 963/1905] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:24:16 2022] DELETE /identity/v3/projects/fe675b007ea14d1d8e3adf534a868eeb => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:16.266984 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0153df6-72d1-4b05-8a8f-e9ecf7c1528b service neutron] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:16.278950 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0153df6-72d1-4b05-8a8f-e9ecf7c1528b service neutron] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:16.359883 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8ed537-f334-4a78-b767-4d7a7e763323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:16.361894 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ed537-f334-4a78-b767-4d7a7e763323 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:16.362503 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ed537-f334-4a78-b767-4d7a7e763323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:16.363417 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ed537-f334-4a78-b767-4d7a7e763323 admin admin] PATH_INFO: `/v3/users/ea821f6e114b473f8f7578273f5454f3` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:16.367439 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8ed537-f334-4a78-b767-4d7a7e763323 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea821f6e114b473f8f7578273f5454f3)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:16.372372 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8ed537-f334-4a78-b767-4d7a7e763323 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:16.387622 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:16.516989 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-2c8ed537-f334-4a78-b767-4d7a7e763323 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea821f6e114b473f8f7578273f5454f3'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:16.581810 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-2c8ed537-f334-4a78-b767-4d7a7e763323 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea821f6e114b473f8f7578273f5454f3'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:16.622898 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-2c8ed537-f334-4a78-b767-4d7a7e763323 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea821f6e114b473f8f7578273f5454f3'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:16.646189 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-2c8ed537-f334-4a78-b767-4d7a7e763323 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea821f6e114b473f8f7578273f5454f3'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:16.651659 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-2c8ed537-f334-4a78-b767-4d7a7e763323 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea821f6e114b473f8f7578273f5454f3'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:16.674546 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 943/1906] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:24:16 2022] DELETE /identity/v3/users/ea821f6e114b473f8f7578273f5454f3 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:17.236904 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad30db-4f44-458c-ab2a-f9dc8b71748d admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:17.240249 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-edad30db-4f44-458c-ab2a-f9dc8b71748d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:17.273366 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad30db-4f44-458c-ab2a-f9dc8b71748d admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:17.375472 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0754e32-ab97-486e-933c-4d9066e7a030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:17.376610 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0754e32-ab97-486e-933c-4d9066e7a030 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:17.376888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0754e32-ab97-486e-933c-4d9066e7a030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:17.377194 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0754e32-ab97-486e-933c-4d9066e7a030 admin admin] PATH_INFO: `/v3/projects/47319ede4d6047259553de024723a3ab` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:17.378379 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0754e32-ab97-486e-933c-4d9066e7a030 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47319ede4d6047259553de024723a3ab)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:17.380515 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0754e32-ab97-486e-933c-4d9066e7a030 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:17.445107 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-a0754e32-ab97-486e-933c-4d9066e7a030 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47319ede4d6047259553de024723a3ab', 'request_id': 'req-a0754e32-ab97-486e-933c-4d9066e7a030'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:17.453179 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-a0754e32-ab97-486e-933c-4d9066e7a030 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47319ede4d6047259553de024723a3ab', 'request_id': 'req-a0754e32-ab97-486e-933c-4d9066e7a030'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:17.463307 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-a0754e32-ab97-486e-933c-4d9066e7a030 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47319ede4d6047259553de024723a3ab', 'request_id': 'req-a0754e32-ab97-486e-933c-4d9066e7a030'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:17.476480 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-a0754e32-ab97-486e-933c-4d9066e7a030 admin admin] The token cache is being invalidate because project 47319ede4d6047259553de024723a3ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:17.476817 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-a0754e32-ab97-486e-933c-4d9066e7a030 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:17.477514 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-a0754e32-ab97-486e-933c-4d9066e7a030 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:17.479306 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 964/1907] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:24:17 2022] DELETE /identity/v3/projects/47319ede4d6047259553de024723a3ab => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:17.484494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931437ea-1af5-4c4a-8991-0abd4ed5f8ec None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:17.485026 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931437ea-1af5-4c4a-8991-0abd4ed5f8ec None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:17.485189 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931437ea-1af5-4c4a-8991-0abd4ed5f8ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:17.505919 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-931437ea-1af5-4c4a-8991-0abd4ed5f8ec None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:17.507433 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-931437ea-1af5-4c4a-8991-0abd4ed5f8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:17.587900 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 944/1908] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:24:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:17.590733 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35ab6e-37e7-42bb-9cb5-821916482d1c None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:17.591085 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35ab6e-37e7-42bb-9cb5-821916482d1c None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:17.591418 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35ab6e-37e7-42bb-9cb5-821916482d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:17.629811 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-0d35ab6e-37e7-42bb-9cb5-821916482d1c None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:17.631774 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-0d35ab6e-37e7-42bb-9cb5-821916482d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:17.653172 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 965/1909] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:24:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:17.657219 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931437ea-1af5-4c4a-8991-0abd4ed5f8ec None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:17.674671 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931437ea-1af5-4c4a-8991-0abd4ed5f8ec None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:17.690165 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d922348-1984-4963-bd17-9eca0997f1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:17.691082 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d922348-1984-4963-bd17-9eca0997f1f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:17.692037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d922348-1984-4963-bd17-9eca0997f1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:17.692427 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d922348-1984-4963-bd17-9eca0997f1f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:17.693328 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d922348-1984-4963-bd17-9eca0997f1f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:24:17.693743 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d922348-1984-4963-bd17-9eca0997f1f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:17.697457 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d922348-1984-4963-bd17-9eca0997f1f8 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:17.740719 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 945/1910] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:24:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:24:17.742528 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35ab6e-37e7-42bb-9cb5-821916482d1c None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:17.750017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:17.750749 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d922348-1984-4963-bd17-9eca0997f1f8 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:17.753411 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-8d922348-1984-4963-bd17-9eca0997f1f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:17.765241 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d35ab6e-37e7-42bb-9cb5-821916482d1c None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:17.771056 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d922348-1984-4963-bd17-9eca0997f1f8 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:17.776295 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b9e272-bb7f-4389-b19c-842dd44fd98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:17.777355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9e272-bb7f-4389-b19c-842dd44fd98f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:17.777724 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9e272-bb7f-4389-b19c-842dd44fd98f admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:17.778010 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b9e272-bb7f-4389-b19c-842dd44fd98f admin admin] PATH_INFO: `/v3/users/19e6611ed2f74ea3b9d053aae16257fe` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:17.779351 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b9e272-bb7f-4389-b19c-842dd44fd98f admin admin] RBAC: Authorizing `identity:delete_user(user_id=19e6611ed2f74ea3b9d053aae16257fe)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:17.780697 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd77cf52-4726-4a65-986e-3e21a1d568e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:17.781548 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b9e272-bb7f-4389-b19c-842dd44fd98f admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:17.781671 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd77cf52-4726-4a65-986e-3e21a1d568e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:17.781920 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd77cf52-4726-4a65-986e-3e21a1d568e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:17.782205 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd77cf52-4726-4a65-986e-3e21a1d568e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:17.782941 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd77cf52-4726-4a65-986e-3e21a1d568e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:17.788546 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd77cf52-4726-4a65-986e-3e21a1d568e5 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:17.846049 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 966/1911] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:24:17 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:17.850768 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd77cf52-4726-4a65-986e-3e21a1d568e5 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:17.869280 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-e2b9e272-bb7f-4389-b19c-842dd44fd98f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19e6611ed2f74ea3b9d053aae16257fe'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:17.887361 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-e2b9e272-bb7f-4389-b19c-842dd44fd98f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19e6611ed2f74ea3b9d053aae16257fe'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:17.889556 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:17.904367 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-e2b9e272-bb7f-4389-b19c-842dd44fd98f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19e6611ed2f74ea3b9d053aae16257fe'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:17.914909 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-e2b9e272-bb7f-4389-b19c-842dd44fd98f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19e6611ed2f74ea3b9d053aae16257fe'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:17.915744 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-e2b9e272-bb7f-4389-b19c-842dd44fd98f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19e6611ed2f74ea3b9d053aae16257fe'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:17.923950 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 946/1912] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:24:17 2022] DELETE /identity/v3/users/19e6611ed2f74ea3b9d053aae16257fe => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:17.928307 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f44553-ebf8-40f1-88cf-7bc6f215cc2f None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:17.928813 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f44553-ebf8-40f1-88cf-7bc6f215cc2f None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:17.929316 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f44553-ebf8-40f1-88cf-7bc6f215cc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:17.970095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-76f44553-ebf8-40f1-88cf-7bc6f215cc2f None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:17.974391 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd77cf52-4726-4a65-986e-3e21a1d568e5 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:17.974561 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-76f44553-ebf8-40f1-88cf-7bc6f215cc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:18.112402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1a30e8-f542-4f65-9f65-71185fe4a4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:18.113497 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a30e8-f542-4f65-9f65-71185fe4a4cc admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:18.113889 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a30e8-f542-4f65-9f65-71185fe4a4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:18.114362 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a30e8-f542-4f65-9f65-71185fe4a4cc admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:18.116412 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1a30e8-f542-4f65-9f65-71185fe4a4cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:18.120401 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 947/1913] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:24:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:24:18.128977 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1a30e8-f542-4f65-9f65-71185fe4a4cc admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:18.131391 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f44553-ebf8-40f1-88cf-7bc6f215cc2f None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:18.133296 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-76f44553-ebf8-40f1-88cf-7bc6f215cc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:18.190956 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 967/1914] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:24:17 2022] POST /identity/v3/users => generated 293 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:18.192107 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1a30e8-f542-4f65-9f65-71185fe4a4cc admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:18.194358 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-7e1a30e8-f542-4f65-9f65-71185fe4a4cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:18.219880 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1a30e8-f542-4f65-9f65-71185fe4a4cc admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:18.231544 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3bb664-c473-4a2b-a86f-e6ba5298e49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:18.231544 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3bb664-c473-4a2b-a86f-e6ba5298e49c admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:18.231544 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3bb664-c473-4a2b-a86f-e6ba5298e49c admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:18.231544 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3bb664-c473-4a2b-a86f-e6ba5298e49c admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:18.231911 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3bb664-c473-4a2b-a86f-e6ba5298e49c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:18.231911 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3bb664-c473-4a2b-a86f-e6ba5298e49c admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:18.246973 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f44553-ebf8-40f1-88cf-7bc6f215cc2f None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:18.252746 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 968/1915] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:24:18.253889 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3bb664-c473-4a2b-a86f-e6ba5298e49c admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:18.254907 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db3e3c-2248-40ea-a568-7bb6d0a7dedb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:18.255582 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db3e3c-2248-40ea-a568-7bb6d0a7dedb service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:18.256413 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db3e3c-2248-40ea-a568-7bb6d0a7dedb service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:18.256723 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db3e3c-2248-40ea-a568-7bb6d0a7dedb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:18.268940 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3bb664-c473-4a2b-a86f-e6ba5298e49c admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:18.275601 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42db3e3c-2248-40ea-a568-7bb6d0a7dedb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:18.277051 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b1ef4f-39a8-4634-a034-ad12cde3952f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:18.277750 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1ef4f-39a8-4634-a034-ad12cde3952f admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:18.277973 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42db3e3c-2248-40ea-a568-7bb6d0a7dedb service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:18.278094 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1ef4f-39a8-4634-a034-ad12cde3952f admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:18.278286 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b1ef4f-39a8-4634-a034-ad12cde3952f admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:18.279013 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b1ef4f-39a8-4634-a034-ad12cde3952f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:18.280725 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b1ef4f-39a8-4634-a034-ad12cde3952f admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:18.296772 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 969/1916] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:18.297075 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 948/1917] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:24:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:24:18.298144 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b1ef4f-39a8-4634-a034-ad12cde3952f admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:18.322622 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b1ef4f-39a8-4634-a034-ad12cde3952f admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:18.330273 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210c7fa-2652-4f5d-9e33-4b5c8667e4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:18.331437 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210c7fa-2652-4f5d-9e33-4b5c8667e4d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:18.331781 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210c7fa-2652-4f5d-9e33-4b5c8667e4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:18.332300 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210c7fa-2652-4f5d-9e33-4b5c8667e4d7 admin admin] PATH_INFO: `/v3/projects/32e6282ee2644af69b1d33070ec10f80/users/f9c253a1d1a947b0993a9f531f5590e0/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:18.353130 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0210c7fa-2652-4f5d-9e33-4b5c8667e4d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32e6282ee2644af69b1d33070ec10f80, user_id=f9c253a1d1a947b0993a9f531f5590e0, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:18.356056 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0210c7fa-2652-4f5d-9e33-4b5c8667e4d7 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:18.382091 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 970/1918] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:24:18 2022] PUT /identity/v3/projects/32e6282ee2644af69b1d33070ec10f80/users/f9c253a1d1a947b0993a9f531f5590e0/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:18.393102 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b4bb5-35b4-4a87-a5e3-2dc0262d7b05 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:18.393428 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b4bb5-35b4-4a87-a5e3-2dc0262d7b05 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:18.393661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b4bb5-35b4-4a87-a5e3-2dc0262d7b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:18.475163 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-fc8b4bb5-35b4-4a87-a5e3-2dc0262d7b05 None None] MFA Rules not processed for user `f9c253a1d1a947b0993a9f531f5590e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:18.478858 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-fc8b4bb5-35b4-4a87-a5e3-2dc0262d7b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:18.596162 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 949/1919] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:24:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:18.600695 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210c7fa-2652-4f5d-9e33-4b5c8667e4d7 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:18.724549 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:18.801160 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210c7fa-2652-4f5d-9e33-4b5c8667e4d7 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:18.819439 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9245dc-c89d-4499-921d-a0adc614826b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:18.821521 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9245dc-c89d-4499-921d-a0adc614826b admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:18.821900 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9245dc-c89d-4499-921d-a0adc614826b admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:18.822228 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9245dc-c89d-4499-921d-a0adc614826b admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:18.823109 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9245dc-c89d-4499-921d-a0adc614826b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:18.832597 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9245dc-c89d-4499-921d-a0adc614826b admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:18.913630 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 971/1920] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:24:18 2022] POST /identity/v3/projects => generated 385 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:18.917546 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b4bb5-35b4-4a87-a5e3-2dc0262d7b05 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:18.937543 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b4bb5-35b4-4a87-a5e3-2dc0262d7b05 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:18.982589 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:19.125761 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9245dc-c89d-4499-921d-a0adc614826b admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:19.127692 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6211e-db89-431c-9bd5-5677f2944aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:19.127838 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-0e9245dc-c89d-4499-921d-a0adc614826b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:19.129013 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6211e-db89-431c-9bd5-5677f2944aab admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:19.129528 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6211e-db89-431c-9bd5-5677f2944aab admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:19.130057 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6211e-db89-431c-9bd5-5677f2944aab admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:19.130957 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f6211e-db89-431c-9bd5-5677f2944aab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:19.136567 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f6211e-db89-431c-9bd5-5677f2944aab admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:19.167401 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9245dc-c89d-4499-921d-a0adc614826b admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:19.179851 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e980ba-b38b-464b-b7cd-0cb676db5744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:19.180921 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e980ba-b38b-464b-b7cd-0cb676db5744 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:19.181272 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e980ba-b38b-464b-b7cd-0cb676db5744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:19.181548 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e980ba-b38b-464b-b7cd-0cb676db5744 admin admin] PATH_INFO: `/v3/projects/53f6e32b4ad44341ba7d0cb848ef6bc5` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:19.184265 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e980ba-b38b-464b-b7cd-0cb676db5744 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53f6e32b4ad44341ba7d0cb848ef6bc5)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:19.186455 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e980ba-b38b-464b-b7cd-0cb676db5744 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:19.187958 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 950/1921] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:24:18 2022] POST /identity/v3/users => generated 299 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:19.191981 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6211e-db89-431c-9bd5-5677f2944aab admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:19.209201 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6211e-db89-431c-9bd5-5677f2944aab admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:19.217074 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5852ceee-49e0-492f-82ee-b51c0791a2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:19.218097 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5852ceee-49e0-492f-82ee-b51c0791a2d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:19.218545 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5852ceee-49e0-492f-82ee-b51c0791a2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:19.219137 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5852ceee-49e0-492f-82ee-b51c0791a2d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:19.220126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5852ceee-49e0-492f-82ee-b51c0791a2d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:19.223760 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5852ceee-49e0-492f-82ee-b51c0791a2d1 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:19.241585 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 951/1922] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:24:19.246156 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5852ceee-49e0-492f-82ee-b51c0791a2d1 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:19.258882 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5852ceee-49e0-492f-82ee-b51c0791a2d1 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:19.264106 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-08e980ba-b38b-464b-b7cd-0cb676db5744 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53f6e32b4ad44341ba7d0cb848ef6bc5', 'request_id': 'req-08e980ba-b38b-464b-b7cd-0cb676db5744'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:19.274582 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-08e980ba-b38b-464b-b7cd-0cb676db5744 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53f6e32b4ad44341ba7d0cb848ef6bc5', 'request_id': 'req-08e980ba-b38b-464b-b7cd-0cb676db5744'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:19.290237 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-08e980ba-b38b-464b-b7cd-0cb676db5744 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53f6e32b4ad44341ba7d0cb848ef6bc5', 'request_id': 'req-08e980ba-b38b-464b-b7cd-0cb676db5744'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:19.323309 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-08e980ba-b38b-464b-b7cd-0cb676db5744 admin admin] The token cache is being invalidate because project 53f6e32b4ad44341ba7d0cb848ef6bc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:19.323853 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-08e980ba-b38b-464b-b7cd-0cb676db5744 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:19.325746 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-08e980ba-b38b-464b-b7cd-0cb676db5744 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:19.328553 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 972/1923] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:24:19 2022] DELETE /identity/v3/projects/53f6e32b4ad44341ba7d0cb848ef6bc5 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:19.330040 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25b698-fba3-4952-bf03-2970dad20fc6 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:19.330485 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25b698-fba3-4952-bf03-2970dad20fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:19.330919 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25b698-fba3-4952-bf03-2970dad20fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:19.369192 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-6c25b698-fba3-4952-bf03-2970dad20fc6 None None] MFA Rules not processed for user `627f0217ac8344318fb58db6dc918649`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:19.370786 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-6c25b698-fba3-4952-bf03-2970dad20fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:19.409000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a357b9-cdf5-4e36-b34e-6e3b8f86844b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:19.409821 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a357b9-cdf5-4e36-b34e-6e3b8f86844b admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:19.410135 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a357b9-cdf5-4e36-b34e-6e3b8f86844b admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:19.410364 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a357b9-cdf5-4e36-b34e-6e3b8f86844b admin admin] PATH_INFO: `/v3/projects/9a184f2a40774a428782b54cbbb565e5/users/6c518139b8ff4d8a81a84cf4531cd972/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:19.436301 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a357b9-cdf5-4e36-b34e-6e3b8f86844b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a184f2a40774a428782b54cbbb565e5, user_id=6c518139b8ff4d8a81a84cf4531cd972, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:19.439207 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a357b9-cdf5-4e36-b34e-6e3b8f86844b admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:19.460759 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 952/1924] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:24:19 2022] PUT /identity/v3/projects/9a184f2a40774a428782b54cbbb565e5/users/6c518139b8ff4d8a81a84cf4531cd972/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:19.460759 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861cacda-dad7-4a04-ac77-b4d50fe42420 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:19.460759 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861cacda-dad7-4a04-ac77-b4d50fe42420 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:19.460759 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861cacda-dad7-4a04-ac77-b4d50fe42420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:19.479936 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-861cacda-dad7-4a04-ac77-b4d50fe42420 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:19.481165 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-861cacda-dad7-4a04-ac77-b4d50fe42420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:19.504141 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 973/1925] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:24:19 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:19.505171 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25b698-fba3-4952-bf03-2970dad20fc6 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:19.509882 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-6c25b698-fba3-4952-bf03-2970dad20fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:19.527008 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25b698-fba3-4952-bf03-2970dad20fc6 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:19.550231 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:19.573461 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 953/1926] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:24:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:19.574380 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861cacda-dad7-4a04-ac77-b4d50fe42420 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:19.575651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-861cacda-dad7-4a04-ac77-b4d50fe42420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:19.596172 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:19.620825 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b62a16a-2d5f-4a54-9ea7-48078b0f2110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:19.621839 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62a16a-2d5f-4a54-9ea7-48078b0f2110 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:19.622644 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62a16a-2d5f-4a54-9ea7-48078b0f2110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:19.623019 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b62a16a-2d5f-4a54-9ea7-48078b0f2110 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:19.624338 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b62a16a-2d5f-4a54-9ea7-48078b0f2110 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:19.627805 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b62a16a-2d5f-4a54-9ea7-48078b0f2110 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:19.641061 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 974/1927] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:19.642248 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6636842-53b7-439a-96ca-29e9fd3674e5 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:19.642529 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6636842-53b7-439a-96ca-29e9fd3674e5 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:19.642797 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6636842-53b7-439a-96ca-29e9fd3674e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:19.656319 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861cacda-dad7-4a04-ac77-b4d50fe42420 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:19.668226 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b44208-4777-4254-b776-6ddda940670d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:19.668945 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b44208-4777-4254-b776-6ddda940670d service nova] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:19.669201 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-d6636842-53b7-439a-96ca-29e9fd3674e5 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:19.669329 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b44208-4777-4254-b776-6ddda940670d service nova] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:19.669550 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b44208-4777-4254-b776-6ddda940670d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:19.671499 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-d6636842-53b7-439a-96ca-29e9fd3674e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:19.671672 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-c5b44208-4777-4254-b776-6ddda940670d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:19.681870 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 975/1928] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:24:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:19.682584 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6636842-53b7-439a-96ca-29e9fd3674e5 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:19.692422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6636842-53b7-439a-96ca-29e9fd3674e5 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:19.697455 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8650424d-da5f-49ef-8757-e08e91561dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:19.698457 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650424d-da5f-49ef-8757-e08e91561dc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:19.698805 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650424d-da5f-49ef-8757-e08e91561dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:19.699139 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650424d-da5f-49ef-8757-e08e91561dc2 admin admin] PATH_INFO: `/v3/projects/9a184f2a40774a428782b54cbbb565e5/users/6c518139b8ff4d8a81a84cf4531cd972/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:19.701138 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8650424d-da5f-49ef-8757-e08e91561dc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a184f2a40774a428782b54cbbb565e5, user_id=6c518139b8ff4d8a81a84cf4531cd972, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:19.703808 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8650424d-da5f-49ef-8757-e08e91561dc2 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:19.717286 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 976/1929] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:24:19 2022] PUT /identity/v3/projects/9a184f2a40774a428782b54cbbb565e5/users/6c518139b8ff4d8a81a84cf4531cd972/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:19.718136 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8650424d-da5f-49ef-8757-e08e91561dc2 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:19.762136 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b44208-4777-4254-b776-6ddda940670d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:19.764074 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b44208-4777-4254-b776-6ddda940670d service nova] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:19.777526 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 954/1930] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:24:19 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:19.779087 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bd28c-67c2-4f4c-a262-a3dbef38d175 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:19.779380 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bd28c-67c2-4f4c-a262-a3dbef38d175 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:19.779789 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bd28c-67c2-4f4c-a262-a3dbef38d175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:19.810766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-8e0bd28c-67c2-4f4c-a262-a3dbef38d175 None None] MFA Rules not processed for user `6c518139b8ff4d8a81a84cf4531cd972`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:19.812158 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-8e0bd28c-67c2-4f4c-a262-a3dbef38d175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:19.817025 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8650424d-da5f-49ef-8757-e08e91561dc2 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:19.859781 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a4785-880d-45b5-8741-60fd476e5cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:19.860643 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a4785-880d-45b5-8741-60fd476e5cd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:19.861053 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a4785-880d-45b5-8741-60fd476e5cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:19.861426 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a4785-880d-45b5-8741-60fd476e5cd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:19.862309 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a4785-880d-45b5-8741-60fd476e5cd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:24:19.862711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a4785-880d-45b5-8741-60fd476e5cd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:19.865218 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745a4785-880d-45b5-8741-60fd476e5cd4 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:19.886000 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 977/1931] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:24:19.890453 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a4785-880d-45b5-8741-60fd476e5cd4 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:19.902818 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a4785-880d-45b5-8741-60fd476e5cd4 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:19.913279 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c44228-4361-411e-9e68-0f115573ef3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:19.914353 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c44228-4361-411e-9e68-0f115573ef3f admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:19.916078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 955/1932] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:24:19 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:19.921888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c44228-4361-411e-9e68-0f115573ef3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:19.921888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c44228-4361-411e-9e68-0f115573ef3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:19.922767 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c44228-4361-411e-9e68-0f115573ef3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:19.924801 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c44228-4361-411e-9e68-0f115573ef3f admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:19.928726 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0bd28c-67c2-4f4c-a262-a3dbef38d175 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:19.931541 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-8e0bd28c-67c2-4f4c-a262-a3dbef38d175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:19.980487 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 978/1933] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:24:19 2022] POST /identity/v3/projects => generated 361 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:24:19.982358 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c44228-4361-411e-9e68-0f115573ef3f admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:20.006453 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:20.028533 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0bd28c-67c2-4f4c-a262-a3dbef38d175 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:20.076251 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c44228-4361-411e-9e68-0f115573ef3f admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:20.084403 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572c3401-1daf-4e53-968e-852e5a466f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:20.085673 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c3401-1daf-4e53-968e-852e5a466f48 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:20.086185 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c3401-1daf-4e53-968e-852e5a466f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:20.086633 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c3401-1daf-4e53-968e-852e5a466f48 admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:20.087563 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572c3401-1daf-4e53-968e-852e5a466f48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:20.095689 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572c3401-1daf-4e53-968e-852e5a466f48 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:20.114673 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff0091-1c82-4098-8a63-fdeff1f30976 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:20.115725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff0091-1c82-4098-8a63-fdeff1f30976 service cinder] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:20.116063 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff0091-1c82-4098-8a63-fdeff1f30976 service cinder] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:20.116699 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff0091-1c82-4098-8a63-fdeff1f30976 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:20.134869 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 979/1934] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:24:19 2022] POST /identity/v3/users => generated 281 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:24:20.146837 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572c3401-1daf-4e53-968e-852e5a466f48 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:20.154296 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:20.163264 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572c3401-1daf-4e53-968e-852e5a466f48 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:20.174777 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7c235b-75c2-4271-a691-1c660a3676d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:20.176536 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c235b-75c2-4271-a691-1c660a3676d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:20.176945 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c235b-75c2-4271-a691-1c660a3676d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:20.182094 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7c235b-75c2-4271-a691-1c660a3676d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:20.184753 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7c235b-75c2-4271-a691-1c660a3676d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:20.186652 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7c235b-75c2-4271-a691-1c660a3676d0 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:20.213217 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 980/1935] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:20.217384 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7c235b-75c2-4271-a691-1c660a3676d0 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:20.222306 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff0091-1c82-4098-8a63-fdeff1f30976 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:20.223923 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff0091-1c82-4098-8a63-fdeff1f30976 service cinder] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:20.233444 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7c235b-75c2-4271-a691-1c660a3676d0 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:20.240110 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 956/1936] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:24:19 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:24:20.240589 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba43e2fc-281e-4c47-b3c8-cd15cdaafdc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:20.241794 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43e2fc-281e-4c47-b3c8-cd15cdaafdc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:20.242133 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43e2fc-281e-4c47-b3c8-cd15cdaafdc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:20.242519 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba43e2fc-281e-4c47-b3c8-cd15cdaafdc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:20.243323 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba43e2fc-281e-4c47-b3c8-cd15cdaafdc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:20.245245 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba43e2fc-281e-4c47-b3c8-cd15cdaafdc8 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:20.280746 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 981/1937] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:20.282663 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff0091-1c82-4098-8a63-fdeff1f30976 service cinder] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:20.304636 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff0091-1c82-4098-8a63-fdeff1f30976 service cinder] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:20.325760 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2600a5a3-4b8e-43fb-b2fc-9eb1e4806233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:20.326749 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600a5a3-4b8e-43fb-b2fc-9eb1e4806233 admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:20.327097 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600a5a3-4b8e-43fb-b2fc-9eb1e4806233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:20.327652 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600a5a3-4b8e-43fb-b2fc-9eb1e4806233 admin admin] PATH_INFO: `/v3/projects/24296619e018463fabd2ac644fe2f01d/users/d0cac8c0efcc4c69b410bb9bd7eafce7/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:20.348692 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2600a5a3-4b8e-43fb-b2fc-9eb1e4806233 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24296619e018463fabd2ac644fe2f01d, user_id=d0cac8c0efcc4c69b410bb9bd7eafce7, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:20.351952 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2600a5a3-4b8e-43fb-b2fc-9eb1e4806233 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:20.411459 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 957/1938] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:24:20 2022] PUT /identity/v3/projects/24296619e018463fabd2ac644fe2f01d/users/d0cac8c0efcc4c69b410bb9bd7eafce7/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:20.412980 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72ea7d-4f7d-450a-85a0-313a52ed9620 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:20.413342 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72ea7d-4f7d-450a-85a0-313a52ed9620 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:20.413662 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72ea7d-4f7d-450a-85a0-313a52ed9620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:20.499039 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-dc72ea7d-4f7d-450a-85a0-313a52ed9620 None None] MFA Rules not processed for user `d0cac8c0efcc4c69b410bb9bd7eafce7`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:20.500715 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-dc72ea7d-4f7d-450a-85a0-313a52ed9620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:20.689240 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 982/1939] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:24:20 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:20.721781 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2600a5a3-4b8e-43fb-b2fc-9eb1e4806233 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:20.832626 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2600a5a3-4b8e-43fb-b2fc-9eb1e4806233 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:20.841336 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a889a-235d-494b-b1dc-bedd7d066ec3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:20.842185 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a889a-235d-494b-b1dc-bedd7d066ec3 service cinder] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:20.845506 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a889a-235d-494b-b1dc-bedd7d066ec3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:20.845506 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a889a-235d-494b-b1dc-bedd7d066ec3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:20.864202 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0a889a-235d-494b-b1dc-bedd7d066ec3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:20.868709 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0a889a-235d-494b-b1dc-bedd7d066ec3 service cinder] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:20.883132 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 958/1940] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:24:20 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:20.906972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72ea7d-4f7d-450a-85a0-313a52ed9620 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:21.051120 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72ea7d-4f7d-450a-85a0-313a52ed9620 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:21.065321 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5140556c-a4ee-4050-a834-df44ebbd4145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:21.066493 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5140556c-a4ee-4050-a834-df44ebbd4145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:21.066865 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5140556c-a4ee-4050-a834-df44ebbd4145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:21.067242 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5140556c-a4ee-4050-a834-df44ebbd4145 admin admin] PATH_INFO: `/v3/users/d0cac8c0efcc4c69b410bb9bd7eafce7` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:21.068825 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5140556c-a4ee-4050-a834-df44ebbd4145 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0cac8c0efcc4c69b410bb9bd7eafce7)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:21.071349 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5140556c-a4ee-4050-a834-df44ebbd4145 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:21.168636 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-5140556c-a4ee-4050-a834-df44ebbd4145 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0cac8c0efcc4c69b410bb9bd7eafce7'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:21.169590 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-5140556c-a4ee-4050-a834-df44ebbd4145 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0cac8c0efcc4c69b410bb9bd7eafce7'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:21.191844 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-5140556c-a4ee-4050-a834-df44ebbd4145 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0cac8c0efcc4c69b410bb9bd7eafce7'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:21.193367 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-5140556c-a4ee-4050-a834-df44ebbd4145 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0cac8c0efcc4c69b410bb9bd7eafce7'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:21.226012 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-5140556c-a4ee-4050-a834-df44ebbd4145 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0cac8c0efcc4c69b410bb9bd7eafce7'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:21.228164 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:21.256625 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 983/1941] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:24:20 2022] DELETE /identity/v3/users/d0cac8c0efcc4c69b410bb9bd7eafce7 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:21.261925 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe91e6-0b2d-4415-8da3-2bca48572148 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:21.262219 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe91e6-0b2d-4415-8da3-2bca48572148 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:21.262496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe91e6-0b2d-4415-8da3-2bca48572148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:21.289568 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-15fe91e6-0b2d-4415-8da3-2bca48572148 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:21.290966 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-15fe91e6-0b2d-4415-8da3-2bca48572148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:21.346188 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5140556c-a4ee-4050-a834-df44ebbd4145 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:21.347772 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-5140556c-a4ee-4050-a834-df44ebbd4145 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:21.363323 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:21.472712 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5140556c-a4ee-4050-a834-df44ebbd4145 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:21.476332 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 959/1942] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:24:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:21.478261 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af9c678-17f4-4ce5-a40c-8a12a8515617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:21.479216 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9c678-17f4-4ce5-a40c-8a12a8515617 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:21.479491 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9c678-17f4-4ce5-a40c-8a12a8515617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:21.479817 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9c678-17f4-4ce5-a40c-8a12a8515617 admin admin] PATH_INFO: `/v3/users/f9c253a1d1a947b0993a9f531f5590e0` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:21.481334 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af9c678-17f4-4ce5-a40c-8a12a8515617 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9c253a1d1a947b0993a9f531f5590e0)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:21.483362 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af9c678-17f4-4ce5-a40c-8a12a8515617 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:21.498315 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fe91e6-0b2d-4415-8da3-2bca48572148 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:21.499816 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-15fe91e6-0b2d-4415-8da3-2bca48572148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:21.563183 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3af9c678-17f4-4ce5-a40c-8a12a8515617 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9c253a1d1a947b0993a9f531f5590e0'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:21.564282 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3af9c678-17f4-4ce5-a40c-8a12a8515617 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9c253a1d1a947b0993a9f531f5590e0'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:21.578241 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3af9c678-17f4-4ce5-a40c-8a12a8515617 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9c253a1d1a947b0993a9f531f5590e0'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:21.579132 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3af9c678-17f4-4ce5-a40c-8a12a8515617 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9c253a1d1a947b0993a9f531f5590e0'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:21.606634 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fe91e6-0b2d-4415-8da3-2bca48572148 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:21.612668 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b1e1a3-07f7-447b-a78d-216e411f648d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:21.613549 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1e1a3-07f7-447b-a78d-216e411f648d service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:21.614000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1e1a3-07f7-447b-a78d-216e411f648d service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:21.614297 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1e1a3-07f7-447b-a78d-216e411f648d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:21.615142 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3af9c678-17f4-4ce5-a40c-8a12a8515617 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9c253a1d1a947b0993a9f531f5590e0'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:21.617707 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-89b1e1a3-07f7-447b-a78d-216e411f648d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:21.636260 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b1e1a3-07f7-447b-a78d-216e411f648d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:21.638135 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b1e1a3-07f7-447b-a78d-216e411f648d service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:21.653195 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 984/1943] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:24:21 2022] DELETE /identity/v3/users/f9c253a1d1a947b0993a9f531f5590e0 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:21.653423 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 960/1944] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:24:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:21.654945 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9e12b-77b6-4c4b-91c4-ed509f326c16 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:21.655241 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9e12b-77b6-4c4b-91c4-ed509f326c16 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:21.655525 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9e12b-77b6-4c4b-91c4-ed509f326c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:21.693367 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-07f9e12b-77b6-4c4b-91c4-ed509f326c16 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:21.695353 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-07f9e12b-77b6-4c4b-91c4-ed509f326c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:21.805744 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 985/1945] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:24:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:24:21.820301 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b1e1a3-07f7-447b-a78d-216e411f648d service neutron] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:21.823946 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-89b1e1a3-07f7-447b-a78d-216e411f648d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:21.849061 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b1e1a3-07f7-447b-a78d-216e411f648d service neutron] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:22.000239 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac30a82-e91f-46b9-ae9b-d27fe6a85a42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:22.001195 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac30a82-e91f-46b9-ae9b-d27fe6a85a42 service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:22.001486 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac30a82-e91f-46b9-ae9b-d27fe6a85a42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:22.001788 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac30a82-e91f-46b9-ae9b-d27fe6a85a42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:22.077830 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac30a82-e91f-46b9-ae9b-d27fe6a85a42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:22.084282 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac30a82-e91f-46b9-ae9b-d27fe6a85a42 service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:22.156565 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 961/1946] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:24:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:24:22.546780 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9e12b-77b6-4c4b-91c4-ed509f326c16 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:22.548441 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-07f9e12b-77b6-4c4b-91c4-ed509f326c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:22.573130 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9e12b-77b6-4c4b-91c4-ed509f326c16 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:22.586940 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0951dd-52e1-4d7b-a814-c1f870dec7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:22.608061 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0951dd-52e1-4d7b-a814-c1f870dec7cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:22.623673 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0951dd-52e1-4d7b-a814-c1f870dec7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:22.623763 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0951dd-52e1-4d7b-a814-c1f870dec7cd admin admin] PATH_INFO: `/v3/projects/24296619e018463fabd2ac644fe2f01d` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:22.627590 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0951dd-52e1-4d7b-a814-c1f870dec7cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=24296619e018463fabd2ac644fe2f01d)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:22.636719 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0951dd-52e1-4d7b-a814-c1f870dec7cd admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:22.641454 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:22.774813 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-9b0951dd-52e1-4d7b-a814-c1f870dec7cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24296619e018463fabd2ac644fe2f01d', 'request_id': 'req-9b0951dd-52e1-4d7b-a814-c1f870dec7cd'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:22.794033 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-9b0951dd-52e1-4d7b-a814-c1f870dec7cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24296619e018463fabd2ac644fe2f01d', 'request_id': 'req-9b0951dd-52e1-4d7b-a814-c1f870dec7cd'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:22.808451 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-9b0951dd-52e1-4d7b-a814-c1f870dec7cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24296619e018463fabd2ac644fe2f01d', 'request_id': 'req-9b0951dd-52e1-4d7b-a814-c1f870dec7cd'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:22.832758 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-9b0951dd-52e1-4d7b-a814-c1f870dec7cd admin admin] The token cache is being invalidate because project 24296619e018463fabd2ac644fe2f01d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:22.833097 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-9b0951dd-52e1-4d7b-a814-c1f870dec7cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:22.834216 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-9b0951dd-52e1-4d7b-a814-c1f870dec7cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:22.837162 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 986/1947] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:24:22 2022] DELETE /identity/v3/projects/24296619e018463fabd2ac644fe2f01d => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:22.842219 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a064f-86c5-48ff-a4e6-f5594caa6557 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:22.842673 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a064f-86c5-48ff-a4e6-f5594caa6557 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:22.842763 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a064f-86c5-48ff-a4e6-f5594caa6557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:22.880397 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-1c5a064f-86c5-48ff-a4e6-f5594caa6557 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:22.882446 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-1c5a064f-86c5-48ff-a4e6-f5594caa6557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:22.929515 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0951dd-52e1-4d7b-a814-c1f870dec7cd admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:22.931525 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-9b0951dd-52e1-4d7b-a814-c1f870dec7cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:23.034929 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 962/1948] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:24:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:23.042921 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76832bec-864c-480f-bc46-b824480ac753 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:23.043300 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76832bec-864c-480f-bc46-b824480ac753 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:23.043762 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76832bec-864c-480f-bc46-b824480ac753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:23.075543 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0951dd-52e1-4d7b-a814-c1f870dec7cd admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:23.080530 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-76832bec-864c-480f-bc46-b824480ac753 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:23.082971 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-76832bec-864c-480f-bc46-b824480ac753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:23.087407 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee1126-d767-402e-b901-6d237bfa9604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:23.088480 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee1126-d767-402e-b901-6d237bfa9604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:23.089895 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee1126-d767-402e-b901-6d237bfa9604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:23.091491 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee1126-d767-402e-b901-6d237bfa9604 admin admin] PATH_INFO: `/v3/projects/32e6282ee2644af69b1d33070ec10f80` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:23.092642 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cee1126-d767-402e-b901-6d237bfa9604 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32e6282ee2644af69b1d33070ec10f80)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:23.095291 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cee1126-d767-402e-b901-6d237bfa9604 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:23.102374 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 963/1949] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:24:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:24:23.106155 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76832bec-864c-480f-bc46-b824480ac753 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:23.117189 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:23.123960 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76832bec-864c-480f-bc46-b824480ac753 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:23.137262 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d649af5-12bb-4ab0-8312-21ecce3d666b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:23.138055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d649af5-12bb-4ab0-8312-21ecce3d666b admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:23.138483 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d649af5-12bb-4ab0-8312-21ecce3d666b admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:23.140611 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d649af5-12bb-4ab0-8312-21ecce3d666b admin admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:23.141485 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d649af5-12bb-4ab0-8312-21ecce3d666b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:24:23.141867 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d649af5-12bb-4ab0-8312-21ecce3d666b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:23.148622 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d649af5-12bb-4ab0-8312-21ecce3d666b admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:23.175733 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 964/1950] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:24:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:24:23.179592 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d649af5-12bb-4ab0-8312-21ecce3d666b admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:23.197517 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d649af5-12bb-4ab0-8312-21ecce3d666b admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:23.205911 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-6cee1126-d767-402e-b901-6d237bfa9604 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32e6282ee2644af69b1d33070ec10f80', 'request_id': 'req-6cee1126-d767-402e-b901-6d237bfa9604'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:23.216721 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-6cee1126-d767-402e-b901-6d237bfa9604 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32e6282ee2644af69b1d33070ec10f80', 'request_id': 'req-6cee1126-d767-402e-b901-6d237bfa9604'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:23.234207 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-6cee1126-d767-402e-b901-6d237bfa9604 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32e6282ee2644af69b1d33070ec10f80', 'request_id': 'req-6cee1126-d767-402e-b901-6d237bfa9604'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:23.254824 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-6cee1126-d767-402e-b901-6d237bfa9604 admin admin] The token cache is being invalidate because project 32e6282ee2644af69b1d33070ec10f80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:23.254824 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-6cee1126-d767-402e-b901-6d237bfa9604 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:23.256515 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-6cee1126-d767-402e-b901-6d237bfa9604 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:23.261287 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 987/1951] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:24:22 2022] DELETE /identity/v3/projects/32e6282ee2644af69b1d33070ec10f80 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:23.267304 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee1126-d767-402e-b901-6d237bfa9604 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:23.270958 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-6cee1126-d767-402e-b901-6d237bfa9604 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:23.344185 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:23.371060 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8496c76d-c7b3-4366-bcfd-41d0f384929e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:23.371746 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496c76d-c7b3-4366-bcfd-41d0f384929e admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:23.372026 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496c76d-c7b3-4366-bcfd-41d0f384929e admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:23.372304 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8496c76d-c7b3-4366-bcfd-41d0f384929e admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:23.373026 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8496c76d-c7b3-4366-bcfd-41d0f384929e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:23.375006 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8496c76d-c7b3-4366-bcfd-41d0f384929e admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:23.402859 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee1126-d767-402e-b901-6d237bfa9604 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:23.409602 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab452c-9011-4deb-b705-b16535661e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:23.410817 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab452c-9011-4deb-b705-b16535661e95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:23.411214 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab452c-9011-4deb-b705-b16535661e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:23.411814 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab452c-9011-4deb-b705-b16535661e95 admin admin] PATH_INFO: `/v3/users/6c518139b8ff4d8a81a84cf4531cd972` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:23.413523 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fab452c-9011-4deb-b705-b16535661e95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c518139b8ff4d8a81a84cf4531cd972)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:23.416077 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fab452c-9011-4deb-b705-b16535661e95 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:23.450371 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 965/1952] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:24:23 2022] POST /identity/v3/projects => generated 363 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:23.454698 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8496c76d-c7b3-4366-bcfd-41d0f384929e admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:23.462147 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-8496c76d-c7b3-4366-bcfd-41d0f384929e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:23.515851 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-8fab452c-9011-4deb-b705-b16535661e95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c518139b8ff4d8a81a84cf4531cd972'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:23.517688 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-8fab452c-9011-4deb-b705-b16535661e95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c518139b8ff4d8a81a84cf4531cd972'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:23.541931 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-8fab452c-9011-4deb-b705-b16535661e95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c518139b8ff4d8a81a84cf4531cd972'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:23.542886 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-8fab452c-9011-4deb-b705-b16535661e95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c518139b8ff4d8a81a84cf4531cd972'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:23.568093 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-8fab452c-9011-4deb-b705-b16535661e95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c518139b8ff4d8a81a84cf4531cd972'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:23.604345 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 988/1953] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:24:23 2022] DELETE /identity/v3/users/6c518139b8ff4d8a81a84cf4531cd972 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:23.611747 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8496c76d-c7b3-4366-bcfd-41d0f384929e admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:23.749435 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d161d37-4f3a-4d40-b686-36c0e5524f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:23.750145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d161d37-4f3a-4d40-b686-36c0e5524f0c admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:23.750434 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d161d37-4f3a-4d40-b686-36c0e5524f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:23.750743 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d161d37-4f3a-4d40-b686-36c0e5524f0c admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:23.751403 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d161d37-4f3a-4d40-b686-36c0e5524f0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:23.761558 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d161d37-4f3a-4d40-b686-36c0e5524f0c admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:23.812380 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 966/1954] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:24:23 2022] POST /identity/v3/users => generated 282 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:23.816353 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab452c-9011-4deb-b705-b16535661e95 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:23.819956 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-8fab452c-9011-4deb-b705-b16535661e95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:23.843933 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab452c-9011-4deb-b705-b16535661e95 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:23.853525 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ec37a-6fef-48e1-b14c-0e6c7752ae76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:23.854371 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ec37a-6fef-48e1-b14c-0e6c7752ae76 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:23.854781 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ec37a-6fef-48e1-b14c-0e6c7752ae76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:23.855159 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ec37a-6fef-48e1-b14c-0e6c7752ae76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:23.856009 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317ec37a-6fef-48e1-b14c-0e6c7752ae76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:23.859171 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317ec37a-6fef-48e1-b14c-0e6c7752ae76 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:23.886642 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 989/1955] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:23.888901 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d161d37-4f3a-4d40-b686-36c0e5524f0c admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:23.905966 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d161d37-4f3a-4d40-b686-36c0e5524f0c admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:23.917204 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200c2535-61e3-4b24-bd12-d05e83ee5432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:23.917907 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c2535-61e3-4b24-bd12-d05e83ee5432 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:23.918632 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c2535-61e3-4b24-bd12-d05e83ee5432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:23.919153 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200c2535-61e3-4b24-bd12-d05e83ee5432 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:23.919899 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200c2535-61e3-4b24-bd12-d05e83ee5432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:23.924453 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200c2535-61e3-4b24-bd12-d05e83ee5432 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:23.953889 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 967/1956] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:23.956582 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ec37a-6fef-48e1-b14c-0e6c7752ae76 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:23.981092 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ec37a-6fef-48e1-b14c-0e6c7752ae76 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:23.999056 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa86db-65e3-4e12-a2c4-f09378a7a8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:24.000060 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa86db-65e3-4e12-a2c4-f09378a7a8ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:24.000364 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa86db-65e3-4e12-a2c4-f09378a7a8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:24.000626 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa86db-65e3-4e12-a2c4-f09378a7a8ad admin admin] PATH_INFO: `/v3/projects/f8da68b918b548bb87f7dfe74383048e/users/2012775e23894b77ae644afbe6dec8fe/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:24.035779 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fa86db-65e3-4e12-a2c4-f09378a7a8ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8da68b918b548bb87f7dfe74383048e, user_id=2012775e23894b77ae644afbe6dec8fe, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:24.040093 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fa86db-65e3-4e12-a2c4-f09378a7a8ad admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:24.069616 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 990/1957] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:24:23 2022] PUT /identity/v3/projects/f8da68b918b548bb87f7dfe74383048e/users/2012775e23894b77ae644afbe6dec8fe/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:24.080566 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a288e6-e37e-4585-ae85-a1365d863282 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:24.080842 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a288e6-e37e-4585-ae85-a1365d863282 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:24.081198 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a288e6-e37e-4585-ae85-a1365d863282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:24.088336 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:24.142441 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-e4a288e6-e37e-4585-ae85-a1365d863282 None None] MFA Rules not processed for user `2012775e23894b77ae644afbe6dec8fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:24.144896 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-e4a288e6-e37e-4585-ae85-a1365d863282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:24.332634 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 968/1958] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:24:24 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:24.350728 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa86db-65e3-4e12-a2c4-f09378a7a8ad admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:24.353487 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-f7fa86db-65e3-4e12-a2c4-f09378a7a8ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:24.526522 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa86db-65e3-4e12-a2c4-f09378a7a8ad admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:24.530761 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f26e1-079c-4d72-94ce-0d77625dcb8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:24.531533 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f26e1-079c-4d72-94ce-0d77625dcb8f service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:24.531819 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f26e1-079c-4d72-94ce-0d77625dcb8f service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:24.532099 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f26e1-079c-4d72-94ce-0d77625dcb8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:24.542438 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953f26e1-079c-4d72-94ce-0d77625dcb8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:24.547730 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953f26e1-079c-4d72-94ce-0d77625dcb8f service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:24.559162 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 991/1959] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:24:24 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:25.088272 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a288e6-e37e-4585-ae85-a1365d863282 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:25.091223 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-e4a288e6-e37e-4585-ae85-a1365d863282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:25.253204 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a288e6-e37e-4585-ae85-a1365d863282 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:25.261935 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de60af-307d-4818-a095-953cbac62f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:25.262844 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de60af-307d-4818-a095-953cbac62f72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:25.263121 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de60af-307d-4818-a095-953cbac62f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:25.263391 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de60af-307d-4818-a095-953cbac62f72 admin admin] PATH_INFO: `/v3/projects/9a184f2a40774a428782b54cbbb565e5` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:25.265117 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de60af-307d-4818-a095-953cbac62f72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a184f2a40774a428782b54cbbb565e5)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:25.267536 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de60af-307d-4818-a095-953cbac62f72 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:25.359694 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-81de60af-307d-4818-a095-953cbac62f72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a184f2a40774a428782b54cbbb565e5', 'request_id': 'req-81de60af-307d-4818-a095-953cbac62f72'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:25.368192 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-81de60af-307d-4818-a095-953cbac62f72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a184f2a40774a428782b54cbbb565e5', 'request_id': 'req-81de60af-307d-4818-a095-953cbac62f72'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:25.382340 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-81de60af-307d-4818-a095-953cbac62f72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a184f2a40774a428782b54cbbb565e5', 'request_id': 'req-81de60af-307d-4818-a095-953cbac62f72'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:25.402523 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-81de60af-307d-4818-a095-953cbac62f72 admin admin] The token cache is being invalidate because project 9a184f2a40774a428782b54cbbb565e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:25.402778 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-81de60af-307d-4818-a095-953cbac62f72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:25.403732 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-81de60af-307d-4818-a095-953cbac62f72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:25.405776 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 969/1960] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:24:25 2022] DELETE /identity/v3/projects/9a184f2a40774a428782b54cbbb565e5 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:25.410838 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495db10a-4b3a-413d-8203-b00c14ff3500 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:25.411717 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495db10a-4b3a-413d-8203-b00c14ff3500 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:25.412151 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495db10a-4b3a-413d-8203-b00c14ff3500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:25.432603 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-495db10a-4b3a-413d-8203-b00c14ff3500 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:25.433996 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-495db10a-4b3a-413d-8203-b00c14ff3500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:25.439954 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:25.519420 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 992/1961] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:24:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:25.528740 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1ebee-ef82-44ed-8595-90dd7bcf59fe None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:25.529051 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1ebee-ef82-44ed-8595-90dd7bcf59fe None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:25.529322 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1ebee-ef82-44ed-8595-90dd7bcf59fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:25.548644 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-aeb1ebee-ef82-44ed-8595-90dd7bcf59fe None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:25.549890 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-aeb1ebee-ef82-44ed-8595-90dd7bcf59fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:25.559047 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 970/1962] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:24:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:25.563789 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495db10a-4b3a-413d-8203-b00c14ff3500 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:25.579884 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495db10a-4b3a-413d-8203-b00c14ff3500 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:25.584939 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4d98b-2830-4478-916f-900a1de27652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:25.585890 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4d98b-2830-4478-916f-900a1de27652 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:25.586176 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4d98b-2830-4478-916f-900a1de27652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:25.586908 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac4d98b-2830-4478-916f-900a1de27652 admin admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:25.587682 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac4d98b-2830-4478-916f-900a1de27652 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:24:25.587995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac4d98b-2830-4478-916f-900a1de27652 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:25.590035 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac4d98b-2830-4478-916f-900a1de27652 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:25.607410 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 993/1963] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:24:25.612499 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb1ebee-ef82-44ed-8595-90dd7bcf59fe None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:25.623911 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb1ebee-ef82-44ed-8595-90dd7bcf59fe None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:25.631445 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e3133a-7dc4-4f09-92da-e0b30529eb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:25.632637 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3133a-7dc4-4f09-92da-e0b30529eb52 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:25.633183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3133a-7dc4-4f09-92da-e0b30529eb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:25.633676 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e3133a-7dc4-4f09-92da-e0b30529eb52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:25.634833 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e3133a-7dc4-4f09-92da-e0b30529eb52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:25.638289 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e3133a-7dc4-4f09-92da-e0b30529eb52 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:25.661381 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:25.684159 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 971/1964] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:24:25 2022] POST /identity/v3/projects => generated 373 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:24:25.689205 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4d98b-2830-4478-916f-900a1de27652 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:25.702051 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac4d98b-2830-4478-916f-900a1de27652 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:25.777996 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc9bd7b-a77d-45d3-bc6e-6491505aa145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:25.779081 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9bd7b-a77d-45d3-bc6e-6491505aa145 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:25.779510 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9bd7b-a77d-45d3-bc6e-6491505aa145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:25.779874 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9bd7b-a77d-45d3-bc6e-6491505aa145 admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:25.780608 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc9bd7b-a77d-45d3-bc6e-6491505aa145 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:25.782738 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc9bd7b-a77d-45d3-bc6e-6491505aa145 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:25.809115 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 994/1965] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:24:25 2022] POST /identity/v3/users => generated 287 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:24:25.813587 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e3133a-7dc4-4f09-92da-e0b30529eb52 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:25.825351 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e3133a-7dc4-4f09-92da-e0b30529eb52 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:25.831344 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cdfcec-ffff-4741-a82e-7ec0a7ccca29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:25.832004 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cdfcec-ffff-4741-a82e-7ec0a7ccca29 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:25.832332 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cdfcec-ffff-4741-a82e-7ec0a7ccca29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:25.832603 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cdfcec-ffff-4741-a82e-7ec0a7ccca29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:25.833400 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cdfcec-ffff-4741-a82e-7ec0a7ccca29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:25.835210 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cdfcec-ffff-4741-a82e-7ec0a7ccca29 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:25.852053 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc9bd7b-a77d-45d3-bc6e-6491505aa145 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:25.852714 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 972/1966] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:24:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:25.863478 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc9bd7b-a77d-45d3-bc6e-6491505aa145 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:25.869658 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a834974-ed85-4101-acc1-a44b54f525fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:25.870444 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a834974-ed85-4101-acc1-a44b54f525fb admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:25.870792 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a834974-ed85-4101-acc1-a44b54f525fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:25.871129 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a834974-ed85-4101-acc1-a44b54f525fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:25.872150 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a834974-ed85-4101-acc1-a44b54f525fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:25.873832 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a834974-ed85-4101-acc1-a44b54f525fb admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:25.892555 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 995/1967] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:24:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:24:25.894372 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cdfcec-ffff-4741-a82e-7ec0a7ccca29 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:25.907313 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cdfcec-ffff-4741-a82e-7ec0a7ccca29 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:25.913073 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd50785d-c9e0-44d7-922c-694bb17fab74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:25.914841 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50785d-c9e0-44d7-922c-694bb17fab74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:25.915267 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50785d-c9e0-44d7-922c-694bb17fab74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:25.915755 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50785d-c9e0-44d7-922c-694bb17fab74 admin admin] PATH_INFO: `/v3/projects/9baee3526f1545f6bc37ca53d963ebed/users/f3137195e2a54f85a97e7ac1d3cb983f/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:25.935526 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd50785d-c9e0-44d7-922c-694bb17fab74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9baee3526f1545f6bc37ca53d963ebed, user_id=f3137195e2a54f85a97e7ac1d3cb983f, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:25.939187 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd50785d-c9e0-44d7-922c-694bb17fab74 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:25.956835 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 973/1968] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:24:25 2022] PUT /identity/v3/projects/9baee3526f1545f6bc37ca53d963ebed/users/f3137195e2a54f85a97e7ac1d3cb983f/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:25.962584 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1eabf-e96b-4226-a3dc-3ceb56354656 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:25.962972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1eabf-e96b-4226-a3dc-3ceb56354656 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:25.963347 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1eabf-e96b-4226-a3dc-3ceb56354656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:25.990563 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-edd1eabf-e96b-4226-a3dc-3ceb56354656 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:25.991772 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-edd1eabf-e96b-4226-a3dc-3ceb56354656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:26.089052 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 996/1969] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:24:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:26.100537 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd50785d-c9e0-44d7-922c-694bb17fab74 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:26.105456 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-cd50785d-c9e0-44d7-922c-694bb17fab74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:26.209311 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd50785d-c9e0-44d7-922c-694bb17fab74 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:26.215672 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a5a700-0dc2-494b-bca1-6b841ae2cef1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:26.236154 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5a700-0dc2-494b-bca1-6b841ae2cef1 service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:26.236451 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5a700-0dc2-494b-bca1-6b841ae2cef1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:26.236731 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5a700-0dc2-494b-bca1-6b841ae2cef1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:26.261225 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a5a700-0dc2-494b-bca1-6b841ae2cef1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:26.262754 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a5a700-0dc2-494b-bca1-6b841ae2cef1 service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:26.321301 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 974/1970] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:24:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:27.550611 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6b25d-a308-40d4-98d9-ea10e25439d6 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:27.551116 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6b25d-a308-40d4-98d9-ea10e25439d6 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:27.551197 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6b25d-a308-40d4-98d9-ea10e25439d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:27.588422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-b4e6b25d-a308-40d4-98d9-ea10e25439d6 None None] MFA Rules not processed for user `f3137195e2a54f85a97e7ac1d3cb983f`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:27.589666 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-b4e6b25d-a308-40d4-98d9-ea10e25439d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:27.593481 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a5a700-0dc2-494b-bca1-6b841ae2cef1 service neutron] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:27.595405 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-c9a5a700-0dc2-494b-bca1-6b841ae2cef1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:27.595956 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:27.610060 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a5a700-0dc2-494b-bca1-6b841ae2cef1 service neutron] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:27.614825 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50914c8d-08df-4f2d-8814-0270c75f127e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:27.616298 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50914c8d-08df-4f2d-8814-0270c75f127e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:27.616669 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50914c8d-08df-4f2d-8814-0270c75f127e admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:27.617041 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50914c8d-08df-4f2d-8814-0270c75f127e admin admin] PATH_INFO: `/v3/users/1b22fa493ae04d5fac13b94cff154eee` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:27.618435 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50914c8d-08df-4f2d-8814-0270c75f127e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b22fa493ae04d5fac13b94cff154eee)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:27.620478 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50914c8d-08df-4f2d-8814-0270c75f127e admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:27.681636 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-50914c8d-08df-4f2d-8814-0270c75f127e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b22fa493ae04d5fac13b94cff154eee'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:27.685012 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 997/1971] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:24:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:27.688827 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6b25d-a308-40d4-98d9-ea10e25439d6 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:27.697068 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-50914c8d-08df-4f2d-8814-0270c75f127e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b22fa493ae04d5fac13b94cff154eee'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:27.701962 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6b25d-a308-40d4-98d9-ea10e25439d6 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:27.707743 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f9689-ac06-482a-aaa2-02ed407d9202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:27.709021 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f9689-ac06-482a-aaa2-02ed407d9202 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:27.709519 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f9689-ac06-482a-aaa2-02ed407d9202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:27.709964 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f9689-ac06-482a-aaa2-02ed407d9202 admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:27.711690 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8f9689-ac06-482a-aaa2-02ed407d9202 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:27.714602 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8f9689-ac06-482a-aaa2-02ed407d9202 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:27.714731 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-50914c8d-08df-4f2d-8814-0270c75f127e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b22fa493ae04d5fac13b94cff154eee'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:27.723918 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-50914c8d-08df-4f2d-8814-0270c75f127e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b22fa493ae04d5fac13b94cff154eee'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:27.724683 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-50914c8d-08df-4f2d-8814-0270c75f127e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b22fa493ae04d5fac13b94cff154eee'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:27.733574 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 975/1972] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:24:27 2022] DELETE /identity/v3/users/1b22fa493ae04d5fac13b94cff154eee => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:24:27.769898 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 998/1973] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:24:27 2022] POST /identity/v3/projects => generated 371 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:27.775180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50914c8d-08df-4f2d-8814-0270c75f127e admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:27.776879 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-50914c8d-08df-4f2d-8814-0270c75f127e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:27.828743 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:27.869426 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50914c8d-08df-4f2d-8814-0270c75f127e admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:27.931943 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e3576-e82b-47ee-aaed-2dadf109b28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:27.932690 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e3576-e82b-47ee-aaed-2dadf109b28e admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:27.933119 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e3576-e82b-47ee-aaed-2dadf109b28e admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:27.933369 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e3576-e82b-47ee-aaed-2dadf109b28e admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:27.934185 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e3576-e82b-47ee-aaed-2dadf109b28e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:27.936444 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e3576-e82b-47ee-aaed-2dadf109b28e admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:27.944545 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f9689-ac06-482a-aaa2-02ed407d9202 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:27.946208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-cf8f9689-ac06-482a-aaa2-02ed407d9202 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:27.961939 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 976/1974] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:24:27 2022] POST /identity/v3/users => generated 292 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:27.962100 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f9689-ac06-482a-aaa2-02ed407d9202 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:27.963873 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e3576-e82b-47ee-aaed-2dadf109b28e admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:27.966571 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430878d8-654f-4141-9753-854c77fd50f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:27.967719 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430878d8-654f-4141-9753-854c77fd50f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:27.968096 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430878d8-654f-4141-9753-854c77fd50f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:27.968436 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430878d8-654f-4141-9753-854c77fd50f4 admin admin] PATH_INFO: `/v3/projects/a0a3ae93f6ab44af93b0538061809b98` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:27.974909 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430878d8-654f-4141-9753-854c77fd50f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0a3ae93f6ab44af93b0538061809b98)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:27.977152 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430878d8-654f-4141-9753-854c77fd50f4 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:27.979248 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e3576-e82b-47ee-aaed-2dadf109b28e admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:27.984825 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4115b01-790c-49b4-9083-dc4e4a5e4b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:27.985522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4115b01-790c-49b4-9083-dc4e4a5e4b5b admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:27.985798 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4115b01-790c-49b4-9083-dc4e4a5e4b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:27.986073 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4115b01-790c-49b4-9083-dc4e4a5e4b5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:27.986805 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4115b01-790c-49b4-9083-dc4e4a5e4b5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:27.988406 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4115b01-790c-49b4-9083-dc4e4a5e4b5b admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:28.000993 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 977/1975] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:28.004089 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4115b01-790c-49b4-9083-dc4e4a5e4b5b admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:28.015949 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4115b01-790c-49b4-9083-dc4e4a5e4b5b admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:28.020847 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42804e98-9da0-43cb-86af-88a66341569d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:28.022018 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42804e98-9da0-43cb-86af-88a66341569d admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:28.022584 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42804e98-9da0-43cb-86af-88a66341569d admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:28.023052 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42804e98-9da0-43cb-86af-88a66341569d admin admin] PATH_INFO: `/v3/projects/9d226aef6baa48daa663cd7f9525ad03/users/f54f3e95dc734dfdb5ed776fc68c9b7f/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:28.036296 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-430878d8-654f-4141-9753-854c77fd50f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0a3ae93f6ab44af93b0538061809b98', 'request_id': 'req-430878d8-654f-4141-9753-854c77fd50f4'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:28.040740 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42804e98-9da0-43cb-86af-88a66341569d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d226aef6baa48daa663cd7f9525ad03, user_id=f54f3e95dc734dfdb5ed776fc68c9b7f, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:28.043503 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42804e98-9da0-43cb-86af-88a66341569d admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:28.043898 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-430878d8-654f-4141-9753-854c77fd50f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0a3ae93f6ab44af93b0538061809b98', 'request_id': 'req-430878d8-654f-4141-9753-854c77fd50f4'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:28.054416 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-430878d8-654f-4141-9753-854c77fd50f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0a3ae93f6ab44af93b0538061809b98', 'request_id': 'req-430878d8-654f-4141-9753-854c77fd50f4'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:28.060286 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 978/1976] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:24:28 2022] PUT /identity/v3/projects/9d226aef6baa48daa663cd7f9525ad03/users/f54f3e95dc734dfdb5ed776fc68c9b7f/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:24:28.064218 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42804e98-9da0-43cb-86af-88a66341569d admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:28.067807 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-430878d8-654f-4141-9753-854c77fd50f4 admin admin] The token cache is being invalidate because project a0a3ae93f6ab44af93b0538061809b98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:28.068189 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-430878d8-654f-4141-9753-854c77fd50f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:28.068999 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-430878d8-654f-4141-9753-854c77fd50f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:28.070784 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 999/1977] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:24:27 2022] DELETE /identity/v3/projects/a0a3ae93f6ab44af93b0538061809b98 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:24:28.076394 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a26b8-c922-4cd3-bd77-76be36821830 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:28.076824 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a26b8-c922-4cd3-bd77-76be36821830 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:28.077025 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a26b8-c922-4cd3-bd77-76be36821830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:28.101693 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-d47a26b8-c922-4cd3-bd77-76be36821830 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:28.103270 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-d47a26b8-c922-4cd3-bd77-76be36821830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:28.161203 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42804e98-9da0-43cb-86af-88a66341569d admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:28.166109 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f53c77-76d6-413c-bc32-330e6add39b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:28.166790 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f53c77-76d6-413c-bc32-330e6add39b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:28.167071 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f53c77-76d6-413c-bc32-330e6add39b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:28.167406 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f53c77-76d6-413c-bc32-330e6add39b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:28.168274 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f53c77-76d6-413c-bc32-330e6add39b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:28.170751 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f53c77-76d6-413c-bc32-330e6add39b6 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:28.186780 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 979/1978] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:24:28 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:24:28.190368 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f53c77-76d6-413c-bc32-330e6add39b6 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:28.192068 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-b0f53c77-76d6-413c-bc32-330e6add39b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:28.202855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1000/1979] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:24:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:24:28.204929 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3863954-aeff-40f2-9b76-2b2f932accd2 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:28.205323 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3863954-aeff-40f2-9b76-2b2f932accd2 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:28.205667 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3863954-aeff-40f2-9b76-2b2f932accd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:28.211334 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f53c77-76d6-413c-bc32-330e6add39b6 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:28.219107 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b2be1b-78f8-46e5-a326-563133053edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:28.220121 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b2be1b-78f8-46e5-a326-563133053edf admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:28.221047 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b2be1b-78f8-46e5-a326-563133053edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:28.221357 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b2be1b-78f8-46e5-a326-563133053edf admin admin] PATH_INFO: `/v3/projects/9d226aef6baa48daa663cd7f9525ad03/users/f54f3e95dc734dfdb5ed776fc68c9b7f/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:28.223381 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b2be1b-78f8-46e5-a326-563133053edf admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d226aef6baa48daa663cd7f9525ad03, user_id=f54f3e95dc734dfdb5ed776fc68c9b7f, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:28.224905 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:28.227849 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b2be1b-78f8-46e5-a326-563133053edf admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:28.230887 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-c3863954-aeff-40f2-9b76-2b2f932accd2 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:28.233125 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-c3863954-aeff-40f2-9b76-2b2f932accd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:28.249382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1001/1980] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:24:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:28.255280 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 980/1981] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:24:28 2022] PUT /identity/v3/projects/9d226aef6baa48daa663cd7f9525ad03/users/f54f3e95dc734dfdb5ed776fc68c9b7f/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:24:28.255539 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3863954-aeff-40f2-9b76-2b2f932accd2 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:28.262444 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3863954-aeff-40f2-9b76-2b2f932accd2 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:28.340560 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e518a423-5aee-414d-be96-545bb253702a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:28.341411 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518a423-5aee-414d-be96-545bb253702a admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:28.341703 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518a423-5aee-414d-be96-545bb253702a admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:28.341971 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e518a423-5aee-414d-be96-545bb253702a admin admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:28.342745 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e518a423-5aee-414d-be96-545bb253702a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:24:28.343048 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e518a423-5aee-414d-be96-545bb253702a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:28.345772 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e518a423-5aee-414d-be96-545bb253702a admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:28.359626 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1002/1982] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:24:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:24:28.363404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b2be1b-78f8-46e5-a326-563133053edf admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:28.379650 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b2be1b-78f8-46e5-a326-563133053edf admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:28.393542 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de962d0d-cac9-4499-92aa-69b858121aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:28.394455 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de962d0d-cac9-4499-92aa-69b858121aa3 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:28.394846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de962d0d-cac9-4499-92aa-69b858121aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:28.395216 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de962d0d-cac9-4499-92aa-69b858121aa3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:28.396168 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de962d0d-cac9-4499-92aa-69b858121aa3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:28.398346 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de962d0d-cac9-4499-92aa-69b858121aa3 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:28.415963 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:28.433902 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 981/1983] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:24:28 2022] POST /identity/v3/projects => generated 365 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:28.435157 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e518a423-5aee-414d-be96-545bb253702a admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:28.448797 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e518a423-5aee-414d-be96-545bb253702a admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:28.516778 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ec769-d6bc-44df-a9bf-1ba3224152c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:28.517481 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ec769-d6bc-44df-a9bf-1ba3224152c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:28.517762 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ec769-d6bc-44df-a9bf-1ba3224152c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:28.518034 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ec769-d6bc-44df-a9bf-1ba3224152c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:28.518726 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ec769-d6bc-44df-a9bf-1ba3224152c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:28.520712 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2ec769-d6bc-44df-a9bf-1ba3224152c3 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:28.545598 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1003/1984] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:24:28 2022] POST /identity/v3/users => generated 283 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:24:28.550053 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de962d0d-cac9-4499-92aa-69b858121aa3 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:28.561356 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de962d0d-cac9-4499-92aa-69b858121aa3 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:28.568863 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0fca8-4d6b-4cef-8dce-8dbf286647da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:28.569916 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0fca8-4d6b-4cef-8dce-8dbf286647da admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:28.570197 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0fca8-4d6b-4cef-8dce-8dbf286647da admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:28.570471 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0fca8-4d6b-4cef-8dce-8dbf286647da admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:28.571447 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd0fca8-4d6b-4cef-8dce-8dbf286647da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:28.574564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd0fca8-4d6b-4cef-8dce-8dbf286647da admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:28.589222 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 982/1985] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:24:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:24:28.594050 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ec769-d6bc-44df-a9bf-1ba3224152c3 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:28.608258 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2ec769-d6bc-44df-a9bf-1ba3224152c3 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:28.615042 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4130bf82-894c-4ae3-882c-9a490d05a33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:28.616123 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130bf82-894c-4ae3-882c-9a490d05a33d admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:28.616678 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130bf82-894c-4ae3-882c-9a490d05a33d admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:28.617192 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130bf82-894c-4ae3-882c-9a490d05a33d admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:28.618730 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4130bf82-894c-4ae3-882c-9a490d05a33d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:28.621053 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4130bf82-894c-4ae3-882c-9a490d05a33d admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:28.637572 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1004/1986] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:24:28 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:24:28.642504 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0fca8-4d6b-4cef-8dce-8dbf286647da admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:28.654172 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0fca8-4d6b-4cef-8dce-8dbf286647da admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:28.659612 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17281dd-8b48-46ab-82fe-a1a96283d382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:28.660647 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17281dd-8b48-46ab-82fe-a1a96283d382 admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:28.660996 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17281dd-8b48-46ab-82fe-a1a96283d382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:28.661278 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17281dd-8b48-46ab-82fe-a1a96283d382 admin admin] PATH_INFO: `/v3/projects/f3f97e1bbc0447158fda5a10f510eb76/users/efdb7aa467e943c9a18d4e12fcd1ca87/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:28.681605 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17281dd-8b48-46ab-82fe-a1a96283d382 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3f97e1bbc0447158fda5a10f510eb76, user_id=efdb7aa467e943c9a18d4e12fcd1ca87, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:28.684814 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17281dd-8b48-46ab-82fe-a1a96283d382 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:28.713083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 983/1987] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:24:28 2022] PUT /identity/v3/projects/f3f97e1bbc0447158fda5a10f510eb76/users/efdb7aa467e943c9a18d4e12fcd1ca87/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:28.717200 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e1fb4-e42e-4988-94a6-eb2e3bef173f None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:28.717611 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e1fb4-e42e-4988-94a6-eb2e3bef173f None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:28.717985 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e1fb4-e42e-4988-94a6-eb2e3bef173f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:28.766581 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-703e1fb4-e42e-4988-94a6-eb2e3bef173f None None] MFA Rules not processed for user `efdb7aa467e943c9a18d4e12fcd1ca87`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:28.768902 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-703e1fb4-e42e-4988-94a6-eb2e3bef173f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:28.866145 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1005/1988] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:24:28 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:28.879651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17281dd-8b48-46ab-82fe-a1a96283d382 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:28.881896 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-b17281dd-8b48-46ab-82fe-a1a96283d382 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:28.953131 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17281dd-8b48-46ab-82fe-a1a96283d382 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:28.956342 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824695da-32b1-4169-ad85-3b2a89e14307 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:28.957435 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824695da-32b1-4169-ad85-3b2a89e14307 service cinder] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:28.957739 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824695da-32b1-4169-ad85-3b2a89e14307 service cinder] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:28.957991 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824695da-32b1-4169-ad85-3b2a89e14307 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:28.970182 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824695da-32b1-4169-ad85-3b2a89e14307 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:28.972048 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824695da-32b1-4169-ad85-3b2a89e14307 service cinder] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:28.984444 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 984/1989] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:24:28 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:29.059322 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec6a724-c9e4-4932-b6ed-b0ce81fee9b9 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:29.059595 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec6a724-c9e4-4932-b6ed-b0ce81fee9b9 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:29.059878 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec6a724-c9e4-4932-b6ed-b0ce81fee9b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:29.066159 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:29.096773 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-eec6a724-c9e4-4932-b6ed-b0ce81fee9b9 None None] MFA Rules not processed for user `f54f3e95dc734dfdb5ed776fc68c9b7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:29.098026 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-eec6a724-c9e4-4932-b6ed-b0ce81fee9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:29.201082 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1006/1990] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:24:29 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:29.213450 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824695da-32b1-4169-ad85-3b2a89e14307 service cinder] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:29.214877 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-824695da-32b1-4169-ad85-3b2a89e14307 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:29.226487 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824695da-32b1-4169-ad85-3b2a89e14307 service cinder] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:29.230203 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e3b09-f7f8-4305-84a0-91dd2f4d9f6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:29.230864 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e3b09-f7f8-4305-84a0-91dd2f4d9f6e service cinder] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:29.231145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e3b09-f7f8-4305-84a0-91dd2f4d9f6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:29.231651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e3b09-f7f8-4305-84a0-91dd2f4d9f6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:29.233348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-106e3b09-f7f8-4305-84a0-91dd2f4d9f6e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:29.317910 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e3b09-f7f8-4305-84a0-91dd2f4d9f6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:29.319959 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e3b09-f7f8-4305-84a0-91dd2f4d9f6e service cinder] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:29.344284 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 985/1991] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:24:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:30.917665 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec6a724-c9e4-4932-b6ed-b0ce81fee9b9 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:30.931032 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec6a724-c9e4-4932-b6ed-b0ce81fee9b9 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:30.942984 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8fb619-06c5-4f50-b460-4e7fe8fa4ee0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:30.943658 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8fb619-06c5-4f50-b460-4e7fe8fa4ee0 service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:30.943938 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8fb619-06c5-4f50-b460-4e7fe8fa4ee0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:30.944211 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8fb619-06c5-4f50-b460-4e7fe8fa4ee0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:30.954284 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8fb619-06c5-4f50-b460-4e7fe8fa4ee0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:30.955748 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8fb619-06c5-4f50-b460-4e7fe8fa4ee0 service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:30.966905 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1007/1992] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:24:30 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:39.336323 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=71,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:39.993632 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e3b09-f7f8-4305-84a0-91dd2f4d9f6e service cinder] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:40.059905 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e3b09-f7f8-4305-84a0-91dd2f4d9f6e service cinder] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:40.065051 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2e34b-d8d6-408d-a277-1da2d0de6b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:40.065971 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2e34b-d8d6-408d-a277-1da2d0de6b63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:40.066247 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2e34b-d8d6-408d-a277-1da2d0de6b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:40.066515 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2e34b-d8d6-408d-a277-1da2d0de6b63 admin admin] PATH_INFO: `/v3/users/efdb7aa467e943c9a18d4e12fcd1ca87` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:40.067754 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2e34b-d8d6-408d-a277-1da2d0de6b63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efdb7aa467e943c9a18d4e12fcd1ca87)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:40.069926 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2e34b-d8d6-408d-a277-1da2d0de6b63 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:40.131632 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-1bf2e34b-d8d6-408d-a277-1da2d0de6b63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efdb7aa467e943c9a18d4e12fcd1ca87'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:40.152217 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-1bf2e34b-d8d6-408d-a277-1da2d0de6b63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efdb7aa467e943c9a18d4e12fcd1ca87'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:40.161866 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-1bf2e34b-d8d6-408d-a277-1da2d0de6b63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efdb7aa467e943c9a18d4e12fcd1ca87'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:40.169656 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-1bf2e34b-d8d6-408d-a277-1da2d0de6b63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efdb7aa467e943c9a18d4e12fcd1ca87'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:40.170433 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-1bf2e34b-d8d6-408d-a277-1da2d0de6b63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efdb7aa467e943c9a18d4e12fcd1ca87'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:40.177004 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 986/1993] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:24:39 2022] DELETE /identity/v3/users/efdb7aa467e943c9a18d4e12fcd1ca87 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:40.181586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a3f93-96c7-46b0-a35a-0a32cf227a94 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:40.182098 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a3f93-96c7-46b0-a35a-0a32cf227a94 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:40.182401 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a3f93-96c7-46b0-a35a-0a32cf227a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:40.204592 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-d99a3f93-96c7-46b0-a35a-0a32cf227a94 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:40.205867 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-d99a3f93-96c7-46b0-a35a-0a32cf227a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:40.282011 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1008/1994] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:24:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:40.291862 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2e34b-d8d6-408d-a277-1da2d0de6b63 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:40.295199 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-1bf2e34b-d8d6-408d-a277-1da2d0de6b63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:40.369766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2e34b-d8d6-408d-a277-1da2d0de6b63 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:40.374277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce17ed0-717f-4bc7-ad9c-40105c2d681e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:40.374943 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce17ed0-717f-4bc7-ad9c-40105c2d681e service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:40.375226 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce17ed0-717f-4bc7-ad9c-40105c2d681e service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:40.375518 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce17ed0-717f-4bc7-ad9c-40105c2d681e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:40.386396 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce17ed0-717f-4bc7-ad9c-40105c2d681e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:40.388659 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce17ed0-717f-4bc7-ad9c-40105c2d681e service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:40.400398 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 987/1995] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:24:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:24:40.705986 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a3f93-96c7-46b0-a35a-0a32cf227a94 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:40.708437 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-d99a3f93-96c7-46b0-a35a-0a32cf227a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:40.722276 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99a3f93-96c7-46b0-a35a-0a32cf227a94 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:40.737999 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034512c7-6703-48ba-9b42-1b01be96e22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:40.739262 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034512c7-6703-48ba-9b42-1b01be96e22c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:40.739588 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034512c7-6703-48ba-9b42-1b01be96e22c admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:40.739871 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034512c7-6703-48ba-9b42-1b01be96e22c admin admin] PATH_INFO: `/v3/projects/f3f97e1bbc0447158fda5a10f510eb76` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:40.743014 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034512c7-6703-48ba-9b42-1b01be96e22c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3f97e1bbc0447158fda5a10f510eb76)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:40.745191 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034512c7-6703-48ba-9b42-1b01be96e22c admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:40.796275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-034512c7-6703-48ba-9b42-1b01be96e22c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3f97e1bbc0447158fda5a10f510eb76', 'request_id': 'req-034512c7-6703-48ba-9b42-1b01be96e22c'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:40.802728 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-034512c7-6703-48ba-9b42-1b01be96e22c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3f97e1bbc0447158fda5a10f510eb76', 'request_id': 'req-034512c7-6703-48ba-9b42-1b01be96e22c'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:40.811734 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-034512c7-6703-48ba-9b42-1b01be96e22c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3f97e1bbc0447158fda5a10f510eb76', 'request_id': 'req-034512c7-6703-48ba-9b42-1b01be96e22c'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:40.818453 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:40.823338 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-034512c7-6703-48ba-9b42-1b01be96e22c admin admin] The token cache is being invalidate because project f3f97e1bbc0447158fda5a10f510eb76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:40.823646 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-034512c7-6703-48ba-9b42-1b01be96e22c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:40.824501 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-034512c7-6703-48ba-9b42-1b01be96e22c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:40.826437 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1009/1996] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:24:40 2022] DELETE /identity/v3/projects/f3f97e1bbc0447158fda5a10f510eb76 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:40.832096 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b7aca-f763-4740-9676-98ac2a155350 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:40.832390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b7aca-f763-4740-9676-98ac2a155350 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:40.832670 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b7aca-f763-4740-9676-98ac2a155350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:40.851342 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-559b7aca-f763-4740-9676-98ac2a155350 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:40.852546 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-559b7aca-f763-4740-9676-98ac2a155350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:40.871832 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:40.919422 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 988/1997] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:24:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:24:40.929540 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb466172-a7ed-4abc-ad7c-7c3877aefbf7 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:40.930080 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb466172-a7ed-4abc-ad7c-7c3877aefbf7 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:40.930614 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb466172-a7ed-4abc-ad7c-7c3877aefbf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:40.952942 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-bb466172-a7ed-4abc-ad7c-7c3877aefbf7 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:40.954221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-bb466172-a7ed-4abc-ad7c-7c3877aefbf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:40.963054 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1010/1998] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:24:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:40.967778 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b7aca-f763-4740-9676-98ac2a155350 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:40.978005 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b7aca-f763-4740-9676-98ac2a155350 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:40.984011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df267f-d6aa-420a-b6ba-45e73cca80fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:40.984679 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df267f-d6aa-420a-b6ba-45e73cca80fe admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:40.984986 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df267f-d6aa-420a-b6ba-45e73cca80fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:40.985285 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df267f-d6aa-420a-b6ba-45e73cca80fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:40.986043 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5df267f-d6aa-420a-b6ba-45e73cca80fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:24:40.986372 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5df267f-d6aa-420a-b6ba-45e73cca80fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:40.988001 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5df267f-d6aa-420a-b6ba-45e73cca80fe admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:41.000718 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 989/1999] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:24:41.005585 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb466172-a7ed-4abc-ad7c-7c3877aefbf7 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:41.016611 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb466172-a7ed-4abc-ad7c-7c3877aefbf7 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:41.022139 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd2621a-37c5-4a7d-b58a-b7c35dcbe220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:41.022807 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2621a-37c5-4a7d-b58a-b7c35dcbe220 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:41.023086 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2621a-37c5-4a7d-b58a-b7c35dcbe220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:41.023365 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2621a-37c5-4a7d-b58a-b7c35dcbe220 admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:41.024069 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd2621a-37c5-4a7d-b58a-b7c35dcbe220 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:41.026088 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd2621a-37c5-4a7d-b58a-b7c35dcbe220 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:41.055126 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1011/2000] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:24:41 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:41.059384 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df267f-d6aa-420a-b6ba-45e73cca80fe admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:41.069488 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df267f-d6aa-420a-b6ba-45e73cca80fe admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:41.133997 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9247db2-18fb-48d1-a6d7-84a64cd2305e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:41.134690 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9247db2-18fb-48d1-a6d7-84a64cd2305e admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:41.134962 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9247db2-18fb-48d1-a6d7-84a64cd2305e admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:41.135229 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9247db2-18fb-48d1-a6d7-84a64cd2305e admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:41.135927 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9247db2-18fb-48d1-a6d7-84a64cd2305e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:41.137958 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9247db2-18fb-48d1-a6d7-84a64cd2305e admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:41.161869 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 990/2001] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:24:41 2022] POST /identity/v3/users => generated 286 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:41.166134 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd2621a-37c5-4a7d-b58a-b7c35dcbe220 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:41.176775 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd2621a-37c5-4a7d-b58a-b7c35dcbe220 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:41.182854 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a2468b-69df-43b8-bb54-5912189a9293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:41.183523 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2468b-69df-43b8-bb54-5912189a9293 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:41.183839 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2468b-69df-43b8-bb54-5912189a9293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:41.184111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a2468b-69df-43b8-bb54-5912189a9293 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:41.184847 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a2468b-69df-43b8-bb54-5912189a9293 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:41.186974 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a2468b-69df-43b8-bb54-5912189a9293 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:41.199692 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1012/2002] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:41.204417 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9247db2-18fb-48d1-a6d7-84a64cd2305e admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:41.218328 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9247db2-18fb-48d1-a6d7-84a64cd2305e admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:41.224314 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c579555-b737-49de-a76a-93600bef63a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:41.225021 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c579555-b737-49de-a76a-93600bef63a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:41.225302 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c579555-b737-49de-a76a-93600bef63a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:41.225570 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c579555-b737-49de-a76a-93600bef63a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:41.226301 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c579555-b737-49de-a76a-93600bef63a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:41.227913 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c579555-b737-49de-a76a-93600bef63a3 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:41.240941 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 991/2003] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:41.245280 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a2468b-69df-43b8-bb54-5912189a9293 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:41.256114 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a2468b-69df-43b8-bb54-5912189a9293 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:41.261439 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c602d829-ee8e-439c-a242-5023918732a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:41.262332 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602d829-ee8e-439c-a242-5023918732a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:41.262613 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602d829-ee8e-439c-a242-5023918732a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:41.262906 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c602d829-ee8e-439c-a242-5023918732a3 admin admin] PATH_INFO: `/v3/projects/c33f84660c0e4c779221c40016d855e5/users/420126299eb747f4a4e81c3cbe46e992/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:41.280930 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c602d829-ee8e-439c-a242-5023918732a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c33f84660c0e4c779221c40016d855e5, user_id=420126299eb747f4a4e81c3cbe46e992, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:41.283064 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c602d829-ee8e-439c-a242-5023918732a3 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:41.301432 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1013/2004] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:24:41 2022] PUT /identity/v3/projects/c33f84660c0e4c779221c40016d855e5/users/420126299eb747f4a4e81c3cbe46e992/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:41.312323 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75d331-f4af-42aa-a149-8031e83d284e None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:41.312585 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75d331-f4af-42aa-a149-8031e83d284e None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:41.312864 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a75d331-f4af-42aa-a149-8031e83d284e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:41.343572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-2a75d331-f4af-42aa-a149-8031e83d284e None None] MFA Rules not processed for user `420126299eb747f4a4e81c3cbe46e992`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:41.344810 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-2a75d331-f4af-42aa-a149-8031e83d284e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:41.419819 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 992/2005] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:24:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:41.425260 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c602d829-ee8e-439c-a242-5023918732a3 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:41.489738 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c602d829-ee8e-439c-a242-5023918732a3 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:41.494833 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3797bf2-164e-4d6f-afb0-12448f18bb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:41.495483 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3797bf2-164e-4d6f-afb0-12448f18bb9d admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:41.495817 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3797bf2-164e-4d6f-afb0-12448f18bb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:41.496094 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3797bf2-164e-4d6f-afb0-12448f18bb9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:41.496823 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3797bf2-164e-4d6f-afb0-12448f18bb9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:41.499011 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3797bf2-164e-4d6f-afb0-12448f18bb9d admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:41.529226 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1014/2006] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:24:41 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:24:41.533278 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a75d331-f4af-42aa-a149-8031e83d284e None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:41.549505 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a75d331-f4af-42aa-a149-8031e83d284e None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:41.575922 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:41.638309 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c49bbb3-b0c0-4564-b4dc-029efde00240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:41.639992 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49bbb3-b0c0-4564-b4dc-029efde00240 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:41.640333 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49bbb3-b0c0-4564-b4dc-029efde00240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:41.640674 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49bbb3-b0c0-4564-b4dc-029efde00240 admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:41.641473 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c49bbb3-b0c0-4564-b4dc-029efde00240 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:41.643588 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c49bbb3-b0c0-4564-b4dc-029efde00240 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:41.668376 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 993/2007] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:24:41 2022] POST /identity/v3/users => generated 293 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:41.673069 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3797bf2-164e-4d6f-afb0-12448f18bb9d admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:41.684273 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3797bf2-164e-4d6f-afb0-12448f18bb9d admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:41.693555 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8542a698-a806-420c-9685-2504a7415b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:41.694365 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542a698-a806-420c-9685-2504a7415b18 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:41.694739 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542a698-a806-420c-9685-2504a7415b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:41.695112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542a698-a806-420c-9685-2504a7415b18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:41.695938 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8542a698-a806-420c-9685-2504a7415b18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:41.697778 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8542a698-a806-420c-9685-2504a7415b18 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:41.713289 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1015/2008] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:41.714587 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c49bbb3-b0c0-4564-b4dc-029efde00240 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:41.730045 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c49bbb3-b0c0-4564-b4dc-029efde00240 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:41.736323 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fdd196-e965-49ec-a495-2d0782d7958c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:41.737280 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fdd196-e965-49ec-a495-2d0782d7958c admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:41.737556 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fdd196-e965-49ec-a495-2d0782d7958c admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:41.737826 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fdd196-e965-49ec-a495-2d0782d7958c admin admin] PATH_INFO: `/v3/projects/08c586c53c6c4280ad4e9b27cce2c35f/users/fe77705136c94dd4942662ede4c21834/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:41.761190 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fdd196-e965-49ec-a495-2d0782d7958c admin admin] RBAC: Authorizing `identity:create_grant(project_id=08c586c53c6c4280ad4e9b27cce2c35f, user_id=fe77705136c94dd4942662ede4c21834, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:41.764008 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fdd196-e965-49ec-a495-2d0782d7958c admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:41.782526 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 994/2009] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:24:41 2022] PUT /identity/v3/projects/08c586c53c6c4280ad4e9b27cce2c35f/users/fe77705136c94dd4942662ede4c21834/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:41.786272 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8542a698-a806-420c-9685-2504a7415b18 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:41.814350 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8542a698-a806-420c-9685-2504a7415b18 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:41.885713 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2160ad87-f714-4610-b5ab-dbc1389adc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:41.886409 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160ad87-f714-4610-b5ab-dbc1389adc8f admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:41.886685 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160ad87-f714-4610-b5ab-dbc1389adc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:41.886960 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160ad87-f714-4610-b5ab-dbc1389adc8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:41.887694 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2160ad87-f714-4610-b5ab-dbc1389adc8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:41.890143 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2160ad87-f714-4610-b5ab-dbc1389adc8f admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:41.902254 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1016/2010] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:41.906741 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fdd196-e965-49ec-a495-2d0782d7958c admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:41.917794 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fdd196-e965-49ec-a495-2d0782d7958c admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:41.923717 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58e205-9226-4fa1-a25d-dba2294e20ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:41.924922 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58e205-9226-4fa1-a25d-dba2294e20ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:41.925256 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58e205-9226-4fa1-a25d-dba2294e20ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:41.925536 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58e205-9226-4fa1-a25d-dba2294e20ba admin admin] PATH_INFO: `/v3/projects/08c586c53c6c4280ad4e9b27cce2c35f/users/fe77705136c94dd4942662ede4c21834/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:41.927996 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab58e205-9226-4fa1-a25d-dba2294e20ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=08c586c53c6c4280ad4e9b27cce2c35f, user_id=fe77705136c94dd4942662ede4c21834, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:41.930871 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab58e205-9226-4fa1-a25d-dba2294e20ba admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:41.944938 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 995/2011] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:24:41 2022] PUT /identity/v3/projects/08c586c53c6c4280ad4e9b27cce2c35f/users/fe77705136c94dd4942662ede4c21834/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:41.958089 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2133bf4b-ca59-4e1a-afc5-67bf32483d81 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:41.958469 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2133bf4b-ca59-4e1a-afc5-67bf32483d81 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:41.959175 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2133bf4b-ca59-4e1a-afc5-67bf32483d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:41.969172 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:42.000762 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-2133bf4b-ca59-4e1a-afc5-67bf32483d81 None None] MFA Rules not processed for user `fe77705136c94dd4942662ede4c21834`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:42.002442 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-2133bf4b-ca59-4e1a-afc5-67bf32483d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:42.099241 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1017/2012] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:24:41 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:42.115951 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58e205-9226-4fa1-a25d-dba2294e20ba admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:42.117940 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-ab58e205-9226-4fa1-a25d-dba2294e20ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:42.182884 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58e205-9226-4fa1-a25d-dba2294e20ba admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:42.186598 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5832b0-bfd3-48fe-8d72-131b85d5f922 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:42.187262 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5832b0-bfd3-48fe-8d72-131b85d5f922 service cinder] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:42.187545 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5832b0-bfd3-48fe-8d72-131b85d5f922 service cinder] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:42.187821 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5832b0-bfd3-48fe-8d72-131b85d5f922 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:42.197679 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5832b0-bfd3-48fe-8d72-131b85d5f922 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:42.199228 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5832b0-bfd3-48fe-8d72-131b85d5f922 service cinder] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:42.210941 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 996/2013] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:24:42 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:42.290729 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2133bf4b-ca59-4e1a-afc5-67bf32483d81 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:42.303761 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2133bf4b-ca59-4e1a-afc5-67bf32483d81 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:42.308501 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c65554-fcee-4617-89ce-8158ef14bd0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:42.309317 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c65554-fcee-4617-89ce-8158ef14bd0a service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:42.309602 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c65554-fcee-4617-89ce-8158ef14bd0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:42.309878 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c65554-fcee-4617-89ce-8158ef14bd0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:42.368862 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c65554-fcee-4617-89ce-8158ef14bd0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:42.370375 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c65554-fcee-4617-89ce-8158ef14bd0a service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:42.380976 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1018/2014] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:24:42 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:24:51.028135 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5832b0-bfd3-48fe-8d72-131b85d5f922 service cinder] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:51.029777 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-3d5832b0-bfd3-48fe-8d72-131b85d5f922 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:51.104891 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5832b0-bfd3-48fe-8d72-131b85d5f922 service cinder] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:51.109596 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c942fe3-c6f5-4be7-8146-7a65b0996f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:51.110793 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c942fe3-c6f5-4be7-8146-7a65b0996f20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:51.111087 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c942fe3-c6f5-4be7-8146-7a65b0996f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:51.111332 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c942fe3-c6f5-4be7-8146-7a65b0996f20 admin admin] PATH_INFO: `/v3/users/f3137195e2a54f85a97e7ac1d3cb983f` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:51.112536 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c942fe3-c6f5-4be7-8146-7a65b0996f20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3137195e2a54f85a97e7ac1d3cb983f)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:51.114616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c942fe3-c6f5-4be7-8146-7a65b0996f20 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:51.130773 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:51.170583 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-9c942fe3-c6f5-4be7-8146-7a65b0996f20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3137195e2a54f85a97e7ac1d3cb983f'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:51.186160 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-9c942fe3-c6f5-4be7-8146-7a65b0996f20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3137195e2a54f85a97e7ac1d3cb983f'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:51.198214 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-9c942fe3-c6f5-4be7-8146-7a65b0996f20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3137195e2a54f85a97e7ac1d3cb983f'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:51.206531 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-9c942fe3-c6f5-4be7-8146-7a65b0996f20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3137195e2a54f85a97e7ac1d3cb983f'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:51.207283 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-9c942fe3-c6f5-4be7-8146-7a65b0996f20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3137195e2a54f85a97e7ac1d3cb983f'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:51.213737 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 997/2015] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:24:51 2022] DELETE /identity/v3/users/f3137195e2a54f85a97e7ac1d3cb983f => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:51.502697 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c65554-fcee-4617-89ce-8158ef14bd0a service cinder] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:51.523393 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c65554-fcee-4617-89ce-8158ef14bd0a service cinder] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:51.600681 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ace4e5-030d-4e43-a914-aae4f7bfc96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:51.601595 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ace4e5-030d-4e43-a914-aae4f7bfc96d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:51.601875 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ace4e5-030d-4e43-a914-aae4f7bfc96d admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:51.602157 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ace4e5-030d-4e43-a914-aae4f7bfc96d admin admin] PATH_INFO: `/v3/projects/9baee3526f1545f6bc37ca53d963ebed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:51.603423 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ace4e5-030d-4e43-a914-aae4f7bfc96d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9baee3526f1545f6bc37ca53d963ebed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:51.605769 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ace4e5-030d-4e43-a914-aae4f7bfc96d admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:51.650020 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:51.664422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-85ace4e5-030d-4e43-a914-aae4f7bfc96d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9baee3526f1545f6bc37ca53d963ebed', 'request_id': 'req-85ace4e5-030d-4e43-a914-aae4f7bfc96d'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:51.670570 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-85ace4e5-030d-4e43-a914-aae4f7bfc96d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9baee3526f1545f6bc37ca53d963ebed', 'request_id': 'req-85ace4e5-030d-4e43-a914-aae4f7bfc96d'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:51.680298 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-85ace4e5-030d-4e43-a914-aae4f7bfc96d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9baee3526f1545f6bc37ca53d963ebed', 'request_id': 'req-85ace4e5-030d-4e43-a914-aae4f7bfc96d'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:51.691521 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-85ace4e5-030d-4e43-a914-aae4f7bfc96d admin admin] The token cache is being invalidate because project 9baee3526f1545f6bc37ca53d963ebed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:51.691692 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-85ace4e5-030d-4e43-a914-aae4f7bfc96d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:51.692529 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-85ace4e5-030d-4e43-a914-aae4f7bfc96d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:51.694169 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1019/2016] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:24:51 2022] DELETE /identity/v3/projects/9baee3526f1545f6bc37ca53d963ebed => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:51.698536 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c942fe3-c6f5-4be7-8146-7a65b0996f20 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:51.700251 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-9c942fe3-c6f5-4be7-8146-7a65b0996f20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:51.714418 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c942fe3-c6f5-4be7-8146-7a65b0996f20 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:51.786044 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0208245d-3504-43af-be72-c8d14340f034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:51.787539 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208245d-3504-43af-be72-c8d14340f034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:51.787954 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208245d-3504-43af-be72-c8d14340f034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:51.788337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0208245d-3504-43af-be72-c8d14340f034 admin admin] PATH_INFO: `/v3/users/f54f3e95dc734dfdb5ed776fc68c9b7f` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:51.790147 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0208245d-3504-43af-be72-c8d14340f034 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f54f3e95dc734dfdb5ed776fc68c9b7f)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:51.792753 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0208245d-3504-43af-be72-c8d14340f034 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:51.851444 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-0208245d-3504-43af-be72-c8d14340f034 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f54f3e95dc734dfdb5ed776fc68c9b7f'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:51.867401 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-0208245d-3504-43af-be72-c8d14340f034 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f54f3e95dc734dfdb5ed776fc68c9b7f'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:51.879303 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-0208245d-3504-43af-be72-c8d14340f034 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f54f3e95dc734dfdb5ed776fc68c9b7f'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:51.886973 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-0208245d-3504-43af-be72-c8d14340f034 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f54f3e95dc734dfdb5ed776fc68c9b7f'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:51.887656 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-0208245d-3504-43af-be72-c8d14340f034 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f54f3e95dc734dfdb5ed776fc68c9b7f'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:51.894425 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 998/2017] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:24:51 2022] DELETE /identity/v3/users/f54f3e95dc734dfdb5ed776fc68c9b7f => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:52.086416 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ace4e5-030d-4e43-a914-aae4f7bfc96d admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:52.094225 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-85ace4e5-030d-4e43-a914-aae4f7bfc96d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:52.110419 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ace4e5-030d-4e43-a914-aae4f7bfc96d admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:52.179835 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2326201b-a9a3-43cf-8f0c-c176ce99bbdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:52.180755 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326201b-a9a3-43cf-8f0c-c176ce99bbdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:52.181136 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326201b-a9a3-43cf-8f0c-c176ce99bbdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:52.181414 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2326201b-a9a3-43cf-8f0c-c176ce99bbdc admin admin] PATH_INFO: `/v3/projects/9d226aef6baa48daa663cd7f9525ad03` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:52.182677 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2326201b-a9a3-43cf-8f0c-c176ce99bbdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d226aef6baa48daa663cd7f9525ad03)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:52.184570 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2326201b-a9a3-43cf-8f0c-c176ce99bbdc admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:52.237568 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-2326201b-a9a3-43cf-8f0c-c176ce99bbdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d226aef6baa48daa663cd7f9525ad03', 'request_id': 'req-2326201b-a9a3-43cf-8f0c-c176ce99bbdc'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:52.243619 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-2326201b-a9a3-43cf-8f0c-c176ce99bbdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d226aef6baa48daa663cd7f9525ad03', 'request_id': 'req-2326201b-a9a3-43cf-8f0c-c176ce99bbdc'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:52.253836 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-2326201b-a9a3-43cf-8f0c-c176ce99bbdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d226aef6baa48daa663cd7f9525ad03', 'request_id': 'req-2326201b-a9a3-43cf-8f0c-c176ce99bbdc'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:52.267790 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-2326201b-a9a3-43cf-8f0c-c176ce99bbdc admin admin] The token cache is being invalidate because project 9d226aef6baa48daa663cd7f9525ad03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:24:52.268037 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-2326201b-a9a3-43cf-8f0c-c176ce99bbdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:52.268893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-2326201b-a9a3-43cf-8f0c-c176ce99bbdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:24:52.270689 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1020/2018] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:24:52 2022] DELETE /identity/v3/projects/9d226aef6baa48daa663cd7f9525ad03 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:52.275957 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673ee5e1-fa09-4e22-85e9-3e8a3df7626e None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:52.276440 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673ee5e1-fa09-4e22-85e9-3e8a3df7626e None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:52.276602 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673ee5e1-fa09-4e22-85e9-3e8a3df7626e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:52.296132 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-673ee5e1-fa09-4e22-85e9-3e8a3df7626e None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:52.297395 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-673ee5e1-fa09-4e22-85e9-3e8a3df7626e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:52.303526 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:52.368881 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 999/2019] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:52.377653 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da88b4-763f-43f8-8029-403aef64a95b None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:52.377944 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da88b4-763f-43f8-8029-403aef64a95b None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:52.378236 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da88b4-763f-43f8-8029-403aef64a95b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:52.397875 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-06da88b4-763f-43f8-8029-403aef64a95b None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:52.399086 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-06da88b4-763f-43f8-8029-403aef64a95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:52.408025 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1021/2020] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:24:52.412677 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673ee5e1-fa09-4e22-85e9-3e8a3df7626e None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:52.423465 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673ee5e1-fa09-4e22-85e9-3e8a3df7626e None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:52.429142 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775950f-0f7d-4707-b3e7-f53927ff6376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:52.429813 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775950f-0f7d-4707-b3e7-f53927ff6376 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:52.430116 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775950f-0f7d-4707-b3e7-f53927ff6376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:52.430424 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775950f-0f7d-4707-b3e7-f53927ff6376 admin admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:52.431178 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8775950f-0f7d-4707-b3e7-f53927ff6376 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:24:52.431481 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8775950f-0f7d-4707-b3e7-f53927ff6376 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:52.433141 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8775950f-0f7d-4707-b3e7-f53927ff6376 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:52.447156 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1000/2021] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:24:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:24:52.451954 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06da88b4-763f-43f8-8029-403aef64a95b None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:52.462920 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06da88b4-763f-43f8-8029-403aef64a95b None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:52.469150 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e51d10-387a-4edf-9d54-3b5058df587e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:52.469940 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e51d10-387a-4edf-9d54-3b5058df587e admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:52.470216 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e51d10-387a-4edf-9d54-3b5058df587e admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:52.470646 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e51d10-387a-4edf-9d54-3b5058df587e admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:52.471357 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e51d10-387a-4edf-9d54-3b5058df587e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:52.473382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e51d10-387a-4edf-9d54-3b5058df587e admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:52.505029 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1022/2022] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:24:52 2022] POST /identity/v3/projects => generated 369 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:52.506853 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775950f-0f7d-4707-b3e7-f53927ff6376 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:52.518838 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775950f-0f7d-4707-b3e7-f53927ff6376 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:52.586783 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9799ae-6c99-4f56-91be-0dbcc76acb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:52.587835 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9799ae-6c99-4f56-91be-0dbcc76acb9a admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:52.588120 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9799ae-6c99-4f56-91be-0dbcc76acb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:52.588404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9799ae-6c99-4f56-91be-0dbcc76acb9a admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:52.589114 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9799ae-6c99-4f56-91be-0dbcc76acb9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:52.591060 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9799ae-6c99-4f56-91be-0dbcc76acb9a admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:52.622756 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1001/2023] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:24:52 2022] POST /identity/v3/users => generated 285 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:24:52.627765 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e51d10-387a-4edf-9d54-3b5058df587e admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:52.638108 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e51d10-387a-4edf-9d54-3b5058df587e admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:52.647407 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadde861-8d9a-4e3c-bb6e-81aaba534a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:52.648116 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadde861-8d9a-4e3c-bb6e-81aaba534a39 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:52.649080 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadde861-8d9a-4e3c-bb6e-81aaba534a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:52.649366 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadde861-8d9a-4e3c-bb6e-81aaba534a39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:52.650110 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadde861-8d9a-4e3c-bb6e-81aaba534a39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:52.651754 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadde861-8d9a-4e3c-bb6e-81aaba534a39 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:52.667787 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1023/2024] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:52.668923 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9799ae-6c99-4f56-91be-0dbcc76acb9a admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:52.681038 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9799ae-6c99-4f56-91be-0dbcc76acb9a admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:52.686516 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7a4464-61af-4f7c-be9f-f5fb7011cef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:52.687406 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a4464-61af-4f7c-be9f-f5fb7011cef3 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:52.687782 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a4464-61af-4f7c-be9f-f5fb7011cef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:52.688163 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7a4464-61af-4f7c-be9f-f5fb7011cef3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:52.689254 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7a4464-61af-4f7c-be9f-f5fb7011cef3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:52.691010 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7a4464-61af-4f7c-be9f-f5fb7011cef3 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:52.704999 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1002/2025] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:24:52.706900 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadde861-8d9a-4e3c-bb6e-81aaba534a39 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:52.718312 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadde861-8d9a-4e3c-bb6e-81aaba534a39 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:52.723354 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16de99-1656-4bf7-8008-ab22e0f437f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:52.724316 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16de99-1656-4bf7-8008-ab22e0f437f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:52.724648 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16de99-1656-4bf7-8008-ab22e0f437f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:52.725081 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16de99-1656-4bf7-8008-ab22e0f437f5 admin admin] PATH_INFO: `/v3/projects/1d499026e5ce4fd1b94c2063e4535957/users/833a3035dcfb46d3b1efde8482a3b743/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:52.728317 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:24:52.743553 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d16de99-1656-4bf7-8008-ab22e0f437f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d499026e5ce4fd1b94c2063e4535957, user_id=833a3035dcfb46d3b1efde8482a3b743, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:52.749688 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d16de99-1656-4bf7-8008-ab22e0f437f5 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:52.767389 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1024/2026] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:24:52 2022] PUT /identity/v3/projects/1d499026e5ce4fd1b94c2063e4535957/users/833a3035dcfb46d3b1efde8482a3b743/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:24:52.778282 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853fb789-f6fd-4b44-b229-3b1a21c1b97f None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:52.778644 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853fb789-f6fd-4b44-b229-3b1a21c1b97f None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:52.778989 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853fb789-f6fd-4b44-b229-3b1a21c1b97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:52.808046 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-853fb789-f6fd-4b44-b229-3b1a21c1b97f None None] MFA Rules not processed for user `833a3035dcfb46d3b1efde8482a3b743`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:24:52.809645 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-853fb789-f6fd-4b44-b229-3b1a21c1b97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:52.877223 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1003/2027] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:24:52 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:24:52.888252 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16de99-1656-4bf7-8008-ab22e0f437f5 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:24:52.890424 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-2d16de99-1656-4bf7-8008-ab22e0f437f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:24:52.956713 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16de99-1656-4bf7-8008-ab22e0f437f5 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:24:52.961623 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9490c075-9798-4b73-adf9-3fce9ed6b91b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3c9c5c8d2e4376b1b70d974d4cb851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:24:52.962348 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490c075-9798-4b73-adf9-3fce9ed6b91b service glance] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:24:52.962692 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490c075-9798-4b73-adf9-3fce9ed6b91b service glance] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:24:52.962976 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9490c075-9798-4b73-adf9-3fce9ed6b91b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:24:52.973640 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9490c075-9798-4b73-adf9-3fce9ed6b91b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:24:52.975167 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9490c075-9798-4b73-adf9-3fce9ed6b91b service glance] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:24:52.984382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1025/2028] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:24:52 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:25:02.873806 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=94,keystone:INSERT=3 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:02.980352 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=32,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:17.425191 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6f29d-ef21-426f-9bac-ca0aba28d3e2 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:17.425765 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6f29d-ef21-426f-9bac-ca0aba28d3e2 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:17.425765 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6f29d-ef21-426f-9bac-ca0aba28d3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:17.457367 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-48a6f29d-ef21-426f-9bac-ca0aba28d3e2 None None] MFA Rules not processed for user `01be6ea7136e45aead352146193f05ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:17.458594 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-48a6f29d-ef21-426f-9bac-ca0aba28d3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:17.559041 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1004/2029] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:25:17 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:27.555835 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=37 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:28.496584 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9490c075-9798-4b73-adf9-3fce9ed6b91b service glance] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:28.498528 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-9490c075-9798-4b73-adf9-3fce9ed6b91b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:28.644074 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9490c075-9798-4b73-adf9-3fce9ed6b91b service glance] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:28.651252 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9cdb44-2217-4719-ae43-cac6867b5e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:28.652531 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9cdb44-2217-4719-ae43-cac6867b5e84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:28.652931 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9cdb44-2217-4719-ae43-cac6867b5e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:28.653450 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9cdb44-2217-4719-ae43-cac6867b5e84 admin admin] PATH_INFO: `/v3/users/2012775e23894b77ae644afbe6dec8fe` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:28.654720 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9cdb44-2217-4719-ae43-cac6867b5e84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2012775e23894b77ae644afbe6dec8fe)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:28.656680 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9cdb44-2217-4719-ae43-cac6867b5e84 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:28.711304 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3d9cdb44-2217-4719-ae43-cac6867b5e84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2012775e23894b77ae644afbe6dec8fe'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:28.712278 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3d9cdb44-2217-4719-ae43-cac6867b5e84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2012775e23894b77ae644afbe6dec8fe'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:28.721304 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3d9cdb44-2217-4719-ae43-cac6867b5e84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2012775e23894b77ae644afbe6dec8fe'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:28.722047 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3d9cdb44-2217-4719-ae43-cac6867b5e84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2012775e23894b77ae644afbe6dec8fe'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:28.738223 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3d9cdb44-2217-4719-ae43-cac6867b5e84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2012775e23894b77ae644afbe6dec8fe'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:28.755375 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1026/2030] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:25:28 2022] DELETE /identity/v3/users/2012775e23894b77ae644afbe6dec8fe => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:28.761569 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a80b56-b536-4dd4-91df-a30ce0e8ab9c None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:28.762027 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a80b56-b536-4dd4-91df-a30ce0e8ab9c None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:28.762123 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a80b56-b536-4dd4-91df-a30ce0e8ab9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:28.796565 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-97a80b56-b536-4dd4-91df-a30ce0e8ab9c None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:28.797858 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-97a80b56-b536-4dd4-91df-a30ce0e8ab9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:28.868775 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1005/2031] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:25:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:28.895258 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9cdb44-2217-4719-ae43-cac6867b5e84 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:28.897124 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-3d9cdb44-2217-4719-ae43-cac6867b5e84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:28.965701 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:29.078800 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9cdb44-2217-4719-ae43-cac6867b5e84 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:29.082884 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9978c581-e44e-4872-afed-dfc5f52392d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:29.083616 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978c581-e44e-4872-afed-dfc5f52392d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:29.083962 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978c581-e44e-4872-afed-dfc5f52392d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:29.084303 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9978c581-e44e-4872-afed-dfc5f52392d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:29.096254 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9978c581-e44e-4872-afed-dfc5f52392d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:29.097816 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9978c581-e44e-4872-afed-dfc5f52392d3 service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:29.117367 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1027/2032] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:25:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:25:29.432354 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a80b56-b536-4dd4-91df-a30ce0e8ab9c None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:29.438613 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-97a80b56-b536-4dd4-91df-a30ce0e8ab9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:29.463333 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a80b56-b536-4dd4-91df-a30ce0e8ab9c None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:29.471253 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a206506-1770-42db-9144-572aa2e998ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:29.472837 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a206506-1770-42db-9144-572aa2e998ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:29.473364 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a206506-1770-42db-9144-572aa2e998ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:29.473856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a206506-1770-42db-9144-572aa2e998ce admin admin] PATH_INFO: `/v3/users/420126299eb747f4a4e81c3cbe46e992` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:29.477544 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a206506-1770-42db-9144-572aa2e998ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=420126299eb747f4a4e81c3cbe46e992)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:29.479612 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a206506-1770-42db-9144-572aa2e998ce admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:29.536810 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-8a206506-1770-42db-9144-572aa2e998ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '420126299eb747f4a4e81c3cbe46e992'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:29.549628 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-8a206506-1770-42db-9144-572aa2e998ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '420126299eb747f4a4e81c3cbe46e992'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:29.559827 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-8a206506-1770-42db-9144-572aa2e998ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '420126299eb747f4a4e81c3cbe46e992'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:29.569369 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-8a206506-1770-42db-9144-572aa2e998ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '420126299eb747f4a4e81c3cbe46e992'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:29.575385 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-8a206506-1770-42db-9144-572aa2e998ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '420126299eb747f4a4e81c3cbe46e992'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:29.583835 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1006/2033] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:25:29 2022] DELETE /identity/v3/users/420126299eb747f4a4e81c3cbe46e992 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:29.588656 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78908e4d-7172-4742-a302-f3701c4ae3db None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:29.589217 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78908e4d-7172-4742-a302-f3701c4ae3db None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:29.589308 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78908e4d-7172-4742-a302-f3701c4ae3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:29.599431 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a206506-1770-42db-9144-572aa2e998ce admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:29.600937 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-8a206506-1770-42db-9144-572aa2e998ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:29.612898 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-78908e4d-7172-4742-a302-f3701c4ae3db None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:29.614158 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-78908e4d-7172-4742-a302-f3701c4ae3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:29.678980 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1028/2034] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:25:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:29.679555 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a206506-1770-42db-9144-572aa2e998ce admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:29.684357 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa3da4-f95b-436a-a17a-221b99176b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:29.685444 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa3da4-f95b-436a-a17a-221b99176b94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:29.685729 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa3da4-f95b-436a-a17a-221b99176b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:29.686007 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa3da4-f95b-436a-a17a-221b99176b94 admin admin] PATH_INFO: `/v3/projects/f8da68b918b548bb87f7dfe74383048e` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:29.687207 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa3da4-f95b-436a-a17a-221b99176b94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8da68b918b548bb87f7dfe74383048e)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:29.688818 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78908e4d-7172-4742-a302-f3701c4ae3db None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:29.689381 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa3da4-f95b-436a-a17a-221b99176b94 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:29.690317 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-78908e4d-7172-4742-a302-f3701c4ae3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:29.699109 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:29.752917 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78908e4d-7172-4742-a302-f3701c4ae3db None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:29.754725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-1efa3da4-f95b-436a-a17a-221b99176b94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8da68b918b548bb87f7dfe74383048e', 'request_id': 'req-1efa3da4-f95b-436a-a17a-221b99176b94'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:29.761731 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-1efa3da4-f95b-436a-a17a-221b99176b94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8da68b918b548bb87f7dfe74383048e', 'request_id': 'req-1efa3da4-f95b-436a-a17a-221b99176b94'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:29.768510 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-1efa3da4-f95b-436a-a17a-221b99176b94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8da68b918b548bb87f7dfe74383048e', 'request_id': 'req-1efa3da4-f95b-436a-a17a-221b99176b94'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:29.787553 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-1efa3da4-f95b-436a-a17a-221b99176b94 admin admin] The token cache is being invalidate because project f8da68b918b548bb87f7dfe74383048e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:25:29.788052 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-1efa3da4-f95b-436a-a17a-221b99176b94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:29.789129 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-1efa3da4-f95b-436a-a17a-221b99176b94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:29.791536 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1007/2035] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:25:29 2022] DELETE /identity/v3/projects/f8da68b918b548bb87f7dfe74383048e => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:25:29.796598 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288fa8eb-83e4-4c51-aeba-d454b4167600 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:29.796882 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288fa8eb-83e4-4c51-aeba-d454b4167600 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:29.797181 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288fa8eb-83e4-4c51-aeba-d454b4167600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:29.807418 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf241e05-112c-469b-81f1-3a8bcb9d7657 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:29.808082 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf241e05-112c-469b-81f1-3a8bcb9d7657 service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:29.808359 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf241e05-112c-469b-81f1-3a8bcb9d7657 service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:29.808630 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf241e05-112c-469b-81f1-3a8bcb9d7657 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:29.810836 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-cf241e05-112c-469b-81f1-3a8bcb9d7657 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:29.816019 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:29.846240 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-288fa8eb-83e4-4c51-aeba-d454b4167600 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:29.847360 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-288fa8eb-83e4-4c51-aeba-d454b4167600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:29.876407 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf241e05-112c-469b-81f1-3a8bcb9d7657 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:29.877898 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf241e05-112c-469b-81f1-3a8bcb9d7657 service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:29.887481 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1029/2036] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:25:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:25:29.907551 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1008/2037] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:25:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:29.916349 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2ba99-113c-4eaa-a5b5-af63dc26b99e None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:29.916642 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2ba99-113c-4eaa-a5b5-af63dc26b99e None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:29.916927 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2ba99-113c-4eaa-a5b5-af63dc26b99e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:29.935882 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-d6b2ba99-113c-4eaa-a5b5-af63dc26b99e None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:29.937139 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-d6b2ba99-113c-4eaa-a5b5-af63dc26b99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:29.945721 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1030/2038] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:25:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:25:29.950200 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288fa8eb-83e4-4c51-aeba-d454b4167600 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:29.960319 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288fa8eb-83e4-4c51-aeba-d454b4167600 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:29.965227 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bc1d3d-bb98-4a7e-98f6-35478df983b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:29.965959 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc1d3d-bb98-4a7e-98f6-35478df983b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:29.966238 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc1d3d-bb98-4a7e-98f6-35478df983b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:29.966516 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc1d3d-bb98-4a7e-98f6-35478df983b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:29.967290 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bc1d3d-bb98-4a7e-98f6-35478df983b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:25:29.967591 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bc1d3d-bb98-4a7e-98f6-35478df983b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:29.969239 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bc1d3d-bb98-4a7e-98f6-35478df983b0 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:29.983372 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1009/2039] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:25:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:25:29.987866 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2ba99-113c-4eaa-a5b5-af63dc26b99e None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:29.998419 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2ba99-113c-4eaa-a5b5-af63dc26b99e None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:30.004109 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092353b2-e463-4a3f-bef5-34b6939dcb45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:30.004774 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092353b2-e463-4a3f-bef5-34b6939dcb45 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:30.005081 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092353b2-e463-4a3f-bef5-34b6939dcb45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:30.005407 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092353b2-e463-4a3f-bef5-34b6939dcb45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:30.006062 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092353b2-e463-4a3f-bef5-34b6939dcb45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:30.008159 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092353b2-e463-4a3f-bef5-34b6939dcb45 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:30.045442 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1031/2040] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:25:29 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:25:30.045923 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bc1d3d-bb98-4a7e-98f6-35478df983b0 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:30.060571 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bc1d3d-bb98-4a7e-98f6-35478df983b0 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:30.123692 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d561b-b484-4f95-9116-50d2300b3914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:30.124336 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d561b-b484-4f95-9116-50d2300b3914 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:30.124614 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d561b-b484-4f95-9116-50d2300b3914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:30.124892 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d561b-b484-4f95-9116-50d2300b3914 admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:30.125624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d561b-b484-4f95-9116-50d2300b3914 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:30.127841 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d561b-b484-4f95-9116-50d2300b3914 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:30.144247 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:30.163813 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1010/2041] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:25:30 2022] POST /identity/v3/users => generated 292 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:25:30.167814 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092353b2-e463-4a3f-bef5-34b6939dcb45 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:30.180450 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092353b2-e463-4a3f-bef5-34b6939dcb45 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:30.185390 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada68d17-05a9-45f3-9d3a-5c6dbfa3e5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:30.186080 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada68d17-05a9-45f3-9d3a-5c6dbfa3e5ca admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:30.186363 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada68d17-05a9-45f3-9d3a-5c6dbfa3e5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:30.186689 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada68d17-05a9-45f3-9d3a-5c6dbfa3e5ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:30.187418 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada68d17-05a9-45f3-9d3a-5c6dbfa3e5ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:30.189115 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada68d17-05a9-45f3-9d3a-5c6dbfa3e5ca admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:30.200034 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1032/2042] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:30.204735 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d561b-b484-4f95-9116-50d2300b3914 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:30.215316 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d561b-b484-4f95-9116-50d2300b3914 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:30.221747 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ee624-2fd6-4dab-b5b1-fd5a0f051fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:30.222611 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ee624-2fd6-4dab-b5b1-fd5a0f051fb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:30.222875 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ee624-2fd6-4dab-b5b1-fd5a0f051fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:30.223196 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ee624-2fd6-4dab-b5b1-fd5a0f051fb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:30.223952 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ee624-2fd6-4dab-b5b1-fd5a0f051fb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:30.225894 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ee624-2fd6-4dab-b5b1-fd5a0f051fb2 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:30.237533 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1011/2043] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:25:30.241859 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada68d17-05a9-45f3-9d3a-5c6dbfa3e5ca admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:30.255204 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada68d17-05a9-45f3-9d3a-5c6dbfa3e5ca admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:30.260413 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee46da-2ac4-40cc-8266-051558820af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:30.261559 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee46da-2ac4-40cc-8266-051558820af5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:30.262133 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee46da-2ac4-40cc-8266-051558820af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:30.262663 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdee46da-2ac4-40cc-8266-051558820af5 admin admin] PATH_INFO: `/v3/projects/4052d211d2a346d8be95d6ac7761defb/users/f0fbd32e5751496e8eaaf3bd03d8f833/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:30.282581 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdee46da-2ac4-40cc-8266-051558820af5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4052d211d2a346d8be95d6ac7761defb, user_id=f0fbd32e5751496e8eaaf3bd03d8f833, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:30.285912 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdee46da-2ac4-40cc-8266-051558820af5 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:30.304700 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1033/2044] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:25:30 2022] PUT /identity/v3/projects/4052d211d2a346d8be95d6ac7761defb/users/f0fbd32e5751496e8eaaf3bd03d8f833/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:25:30.314696 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e96f5-7efc-45b4-8353-450dff11ceaa None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:30.315030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e96f5-7efc-45b4-8353-450dff11ceaa None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:30.315366 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e96f5-7efc-45b4-8353-450dff11ceaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:30.337248 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee46da-2ac4-40cc-8266-051558820af5 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:30.337367 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-0a4e96f5-7efc-45b4-8353-450dff11ceaa None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:30.338910 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-0a4e96f5-7efc-45b4-8353-450dff11ceaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:30.339256 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-cdee46da-2ac4-40cc-8266-051558820af5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:30.413681 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdee46da-2ac4-40cc-8266-051558820af5 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:30.418433 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3685d9dd-df23-4a97-995d-a3c8532a37c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:30.419373 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3685d9dd-df23-4a97-995d-a3c8532a37c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:30.419690 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3685d9dd-df23-4a97-995d-a3c8532a37c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:30.419924 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3685d9dd-df23-4a97-995d-a3c8532a37c8 admin admin] PATH_INFO: `/v3/projects/c33f84660c0e4c779221c40016d855e5` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:30.421176 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3685d9dd-df23-4a97-995d-a3c8532a37c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c33f84660c0e4c779221c40016d855e5)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:30.426256 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1012/2045] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:25:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:30.426404 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3685d9dd-df23-4a97-995d-a3c8532a37c8 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:30.433254 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e96f5-7efc-45b4-8353-450dff11ceaa None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:30.435403 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-0a4e96f5-7efc-45b4-8353-450dff11ceaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:30.482093 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:30.495238 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3685d9dd-df23-4a97-995d-a3c8532a37c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c33f84660c0e4c779221c40016d855e5', 'request_id': 'req-3685d9dd-df23-4a97-995d-a3c8532a37c8'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:30.504324 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3685d9dd-df23-4a97-995d-a3c8532a37c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c33f84660c0e4c779221c40016d855e5', 'request_id': 'req-3685d9dd-df23-4a97-995d-a3c8532a37c8'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:30.517246 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e96f5-7efc-45b4-8353-450dff11ceaa None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:30.527314 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3685d9dd-df23-4a97-995d-a3c8532a37c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c33f84660c0e4c779221c40016d855e5', 'request_id': 'req-3685d9dd-df23-4a97-995d-a3c8532a37c8'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:30.543298 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3685d9dd-df23-4a97-995d-a3c8532a37c8 admin admin] The token cache is being invalidate because project c33f84660c0e4c779221c40016d855e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:25:30.543298 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3685d9dd-df23-4a97-995d-a3c8532a37c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:30.545361 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3685d9dd-df23-4a97-995d-a3c8532a37c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:30.557585 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1034/2046] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:25:30 2022] DELETE /identity/v3/projects/c33f84660c0e4c779221c40016d855e5 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:25:30.562542 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3685d9dd-df23-4a97-995d-a3c8532a37c8 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:30.564113 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-3685d9dd-df23-4a97-995d-a3c8532a37c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:30.579277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb463d5b-7c81-49c4-93f0-2fd84dd63602 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:30.580211 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb463d5b-7c81-49c4-93f0-2fd84dd63602 service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:30.580771 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb463d5b-7c81-49c4-93f0-2fd84dd63602 service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:30.581263 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb463d5b-7c81-49c4-93f0-2fd84dd63602 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:30.584438 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-bb463d5b-7c81-49c4-93f0-2fd84dd63602 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:30.663467 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3685d9dd-df23-4a97-995d-a3c8532a37c8 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:30.668362 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15db5e1-50d2-45dc-b408-70f78b5afe07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:30.669607 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15db5e1-50d2-45dc-b408-70f78b5afe07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:30.679738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:30.681324 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15db5e1-50d2-45dc-b408-70f78b5afe07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:30.681660 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15db5e1-50d2-45dc-b408-70f78b5afe07 admin admin] PATH_INFO: `/v3/users/fe77705136c94dd4942662ede4c21834` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:30.686248 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15db5e1-50d2-45dc-b408-70f78b5afe07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe77705136c94dd4942662ede4c21834)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:30.688257 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15db5e1-50d2-45dc-b408-70f78b5afe07 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:30.720873 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb463d5b-7c81-49c4-93f0-2fd84dd63602 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:30.722834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb463d5b-7c81-49c4-93f0-2fd84dd63602 service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:30.755816 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1013/2047] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:25:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:25:30.793706 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-c15db5e1-50d2-45dc-b408-70f78b5afe07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe77705136c94dd4942662ede4c21834'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:30.794480 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-c15db5e1-50d2-45dc-b408-70f78b5afe07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe77705136c94dd4942662ede4c21834'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:30.803341 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-c15db5e1-50d2-45dc-b408-70f78b5afe07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe77705136c94dd4942662ede4c21834'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:30.804006 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-c15db5e1-50d2-45dc-b408-70f78b5afe07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe77705136c94dd4942662ede4c21834'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:30.816990 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-c15db5e1-50d2-45dc-b408-70f78b5afe07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe77705136c94dd4942662ede4c21834'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:30.830812 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1035/2048] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:25:30 2022] DELETE /identity/v3/users/fe77705136c94dd4942662ede4c21834 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:31.231361 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb463d5b-7c81-49c4-93f0-2fd84dd63602 service neutron] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:31.244715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb463d5b-7c81-49c4-93f0-2fd84dd63602 service neutron] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:31.342942 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e86e99b-e76f-4012-8ccc-e0535cb06ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:31.343861 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86e99b-e76f-4012-8ccc-e0535cb06ec0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:31.344143 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86e99b-e76f-4012-8ccc-e0535cb06ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:31.344419 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86e99b-e76f-4012-8ccc-e0535cb06ec0 admin admin] PATH_INFO: `/v3/projects/08c586c53c6c4280ad4e9b27cce2c35f` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:31.345790 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e86e99b-e76f-4012-8ccc-e0535cb06ec0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08c586c53c6c4280ad4e9b27cce2c35f)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:31.348041 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e86e99b-e76f-4012-8ccc-e0535cb06ec0 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:31.450493 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-3e86e99b-e76f-4012-8ccc-e0535cb06ec0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08c586c53c6c4280ad4e9b27cce2c35f', 'request_id': 'req-3e86e99b-e76f-4012-8ccc-e0535cb06ec0'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:31.459647 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-3e86e99b-e76f-4012-8ccc-e0535cb06ec0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08c586c53c6c4280ad4e9b27cce2c35f', 'request_id': 'req-3e86e99b-e76f-4012-8ccc-e0535cb06ec0'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:31.473155 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-3e86e99b-e76f-4012-8ccc-e0535cb06ec0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08c586c53c6c4280ad4e9b27cce2c35f', 'request_id': 'req-3e86e99b-e76f-4012-8ccc-e0535cb06ec0'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:31.490724 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-3e86e99b-e76f-4012-8ccc-e0535cb06ec0 admin admin] The token cache is being invalidate because project 08c586c53c6c4280ad4e9b27cce2c35f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:25:31.491020 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-3e86e99b-e76f-4012-8ccc-e0535cb06ec0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:31.492288 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-3e86e99b-e76f-4012-8ccc-e0535cb06ec0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:31.497513 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1014/2049] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:25:31 2022] DELETE /identity/v3/projects/08c586c53c6c4280ad4e9b27cce2c35f => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:31.499795 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa49504-5125-46c7-a94e-16db2f41b7af None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:31.500159 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa49504-5125-46c7-a94e-16db2f41b7af None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:31.500436 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa49504-5125-46c7-a94e-16db2f41b7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:31.523970 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-2aa49504-5125-46c7-a94e-16db2f41b7af None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:31.525250 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-2aa49504-5125-46c7-a94e-16db2f41b7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:31.590368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:31.611932 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1036/2050] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:25:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:31.621125 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb959e-563b-4e0c-ae57-9f7a6ed1f0bb None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:31.621414 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb959e-563b-4e0c-ae57-9f7a6ed1f0bb None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:31.621895 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb959e-563b-4e0c-ae57-9f7a6ed1f0bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:31.644674 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-9cdb959e-563b-4e0c-ae57-9f7a6ed1f0bb None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:31.645958 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-9cdb959e-563b-4e0c-ae57-9f7a6ed1f0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:31.659435 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1015/2051] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:25:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:31.660387 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa49504-5125-46c7-a94e-16db2f41b7af None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:31.670629 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa49504-5125-46c7-a94e-16db2f41b7af None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:31.676571 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84850e-3a1d-4b11-bbe8-29d9237051a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:31.677540 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84850e-3a1d-4b11-bbe8-29d9237051a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:31.677881 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84850e-3a1d-4b11-bbe8-29d9237051a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:31.678167 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84850e-3a1d-4b11-bbe8-29d9237051a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:31.679356 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a84850e-3a1d-4b11-bbe8-29d9237051a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:25:31.679654 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a84850e-3a1d-4b11-bbe8-29d9237051a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:31.681322 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a84850e-3a1d-4b11-bbe8-29d9237051a7 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:31.695639 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1037/2052] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:25:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:25:31.699693 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdb959e-563b-4e0c-ae57-9f7a6ed1f0bb None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:31.714204 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdb959e-563b-4e0c-ae57-9f7a6ed1f0bb None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:31.725437 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f34b81-d674-4721-9248-1b54ddba3912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:31.726205 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f34b81-d674-4721-9248-1b54ddba3912 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:31.727410 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f34b81-d674-4721-9248-1b54ddba3912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:31.727803 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f34b81-d674-4721-9248-1b54ddba3912 admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:31.729911 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f34b81-d674-4721-9248-1b54ddba3912 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:31.738991 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f34b81-d674-4721-9248-1b54ddba3912 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:31.791522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1016/2053] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:25:31 2022] POST /identity/v3/projects => generated 369 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:25:31.796097 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84850e-3a1d-4b11-bbe8-29d9237051a7 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:31.815056 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84850e-3a1d-4b11-bbe8-29d9237051a7 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:31.892807 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753cd437-1f9f-4af3-93d5-6ba36385d050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:31.893677 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753cd437-1f9f-4af3-93d5-6ba36385d050 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:31.894010 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753cd437-1f9f-4af3-93d5-6ba36385d050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:31.894325 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753cd437-1f9f-4af3-93d5-6ba36385d050 admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:31.895135 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753cd437-1f9f-4af3-93d5-6ba36385d050 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:31.898626 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753cd437-1f9f-4af3-93d5-6ba36385d050 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:31.962819 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1038/2054] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:25:31 2022] POST /identity/v3/users => generated 285 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:25:31.967073 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f34b81-d674-4721-9248-1b54ddba3912 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:31.982872 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f34b81-d674-4721-9248-1b54ddba3912 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:31.998417 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef987355-4bfd-4e26-ba12-ad082853a2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:32.000714 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef987355-4bfd-4e26-ba12-ad082853a2e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:32.001171 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef987355-4bfd-4e26-ba12-ad082853a2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:32.001458 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef987355-4bfd-4e26-ba12-ad082853a2e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:32.003081 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef987355-4bfd-4e26-ba12-ad082853a2e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:32.007517 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef987355-4bfd-4e26-ba12-ad082853a2e5 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:32.045496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1017/2055] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:32.054065 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753cd437-1f9f-4af3-93d5-6ba36385d050 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:32.077285 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753cd437-1f9f-4af3-93d5-6ba36385d050 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:32.102104 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a1426-639b-46a0-b0b3-7539461990bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:32.102808 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a1426-639b-46a0-b0b3-7539461990bf admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:32.103136 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a1426-639b-46a0-b0b3-7539461990bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:32.103367 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a1426-639b-46a0-b0b3-7539461990bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:32.104107 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a1426-639b-46a0-b0b3-7539461990bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:32.107435 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a1426-639b-46a0-b0b3-7539461990bf admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:32.150788 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1039/2056] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:32.155791 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef987355-4bfd-4e26-ba12-ad082853a2e5 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:32.205449 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef987355-4bfd-4e26-ba12-ad082853a2e5 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:32.220637 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa039245-cf10-48c8-877b-7d5c5e2a6c72 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:32.221033 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa039245-cf10-48c8-877b-7d5c5e2a6c72 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:32.221324 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa039245-cf10-48c8-877b-7d5c5e2a6c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:32.223260 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b6577-fdb1-4ea2-9e6e-4fa2010fc333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:32.224256 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b6577-fdb1-4ea2-9e6e-4fa2010fc333 admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:32.224628 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b6577-fdb1-4ea2-9e6e-4fa2010fc333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:32.225002 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b6577-fdb1-4ea2-9e6e-4fa2010fc333 admin admin] PATH_INFO: `/v3/projects/750d1d9f4e714ac7a31641006ecfd89b/users/bb82e297d6e5438898d5e5fad1756e79/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:32.254961 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372b6577-fdb1-4ea2-9e6e-4fa2010fc333 admin admin] RBAC: Authorizing `identity:create_grant(project_id=750d1d9f4e714ac7a31641006ecfd89b, user_id=bb82e297d6e5438898d5e5fad1756e79, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:32.259236 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372b6577-fdb1-4ea2-9e6e-4fa2010fc333 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:32.304212 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1018/2057] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:25:32 2022] PUT /identity/v3/projects/750d1d9f4e714ac7a31641006ecfd89b/users/bb82e297d6e5438898d5e5fad1756e79/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:32.304393 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-aa039245-cf10-48c8-877b-7d5c5e2a6c72 None None] MFA Rules not processed for user `f0fbd32e5751496e8eaaf3bd03d8f833`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:32.305765 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-aa039245-cf10-48c8-877b-7d5c5e2a6c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:32.320000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f37df-ea32-4acb-9f47-8da50cd1181f None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:32.320000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f37df-ea32-4acb-9f47-8da50cd1181f None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:32.320230 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8f37df-ea32-4acb-9f47-8da50cd1181f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:32.343910 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:32.353374 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-de8f37df-ea32-4acb-9f47-8da50cd1181f None None] MFA Rules not processed for user `bb82e297d6e5438898d5e5fad1756e79`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:32.355966 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-de8f37df-ea32-4acb-9f47-8da50cd1181f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:32.442730 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1040/2058] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:25:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:32.454921 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa039245-cf10-48c8-877b-7d5c5e2a6c72 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:32.456683 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1019/2059] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:25:32 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:32.459893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-aa039245-cf10-48c8-877b-7d5c5e2a6c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:32.461255 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8f37df-ea32-4acb-9f47-8da50cd1181f None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:32.521711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:32.547508 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa039245-cf10-48c8-877b-7d5c5e2a6c72 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:32.557142 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8f37df-ea32-4acb-9f47-8da50cd1181f None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:32.557274 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0998485-0093-4efc-8578-c25e33d6128f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:32.557274 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0998485-0093-4efc-8578-c25e33d6128f service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:32.557274 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0998485-0093-4efc-8578-c25e33d6128f service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:32.557274 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0998485-0093-4efc-8578-c25e33d6128f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:32.558467 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986e41c-21bf-4429-a400-401f0c0543e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:32.559480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986e41c-21bf-4429-a400-401f0c0543e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:32.559828 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986e41c-21bf-4429-a400-401f0c0543e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:32.560114 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986e41c-21bf-4429-a400-401f0c0543e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:32.560877 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c986e41c-21bf-4429-a400-401f0c0543e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:32.563202 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c986e41c-21bf-4429-a400-401f0c0543e1 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:32.568939 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0998485-0093-4efc-8578-c25e33d6128f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:32.570412 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0998485-0093-4efc-8578-c25e33d6128f service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:32.581037 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1041/2060] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:25:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:25:32.594555 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1020/2061] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:25:32 2022] POST /identity/v3/projects => generated 367 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:25:32.598540 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0998485-0093-4efc-8578-c25e33d6128f service cinder] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:32.611775 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0998485-0093-4efc-8578-c25e33d6128f service cinder] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:32.670200 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d44fc4d-11de-4fcf-9309-aeaa77a9b023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:32.670893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44fc4d-11de-4fcf-9309-aeaa77a9b023 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:32.671464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44fc4d-11de-4fcf-9309-aeaa77a9b023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:32.671464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d44fc4d-11de-4fcf-9309-aeaa77a9b023 admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:32.672097 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d44fc4d-11de-4fcf-9309-aeaa77a9b023 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:32.674133 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d44fc4d-11de-4fcf-9309-aeaa77a9b023 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:32.696993 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1042/2062] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:25:32 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:25:32.700166 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986e41c-21bf-4429-a400-401f0c0543e1 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:32.711410 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986e41c-21bf-4429-a400-401f0c0543e1 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:32.716199 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222c13ad-38b4-4e28-b51d-1d4024d7f746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:32.716884 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c13ad-38b4-4e28-b51d-1d4024d7f746 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:32.717204 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c13ad-38b4-4e28-b51d-1d4024d7f746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:32.717480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222c13ad-38b4-4e28-b51d-1d4024d7f746 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:32.718213 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222c13ad-38b4-4e28-b51d-1d4024d7f746 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:32.719881 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222c13ad-38b4-4e28-b51d-1d4024d7f746 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:32.731145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1021/2063] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:32.735862 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d44fc4d-11de-4fcf-9309-aeaa77a9b023 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:32.746335 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d44fc4d-11de-4fcf-9309-aeaa77a9b023 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:32.751128 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ab3584-3157-4f61-a264-a8c28d3c1e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:32.752082 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ab3584-3157-4f61-a264-a8c28d3c1e1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:32.752297 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ab3584-3157-4f61-a264-a8c28d3c1e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:32.752570 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ab3584-3157-4f61-a264-a8c28d3c1e1f admin admin] PATH_INFO: `/v3/projects/979a5be6112b45cfb8ab0727d9003929/users/6da2b52b9fcd413d8d0160032e3aeedb/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:32.768836 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ab3584-3157-4f61-a264-a8c28d3c1e1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=979a5be6112b45cfb8ab0727d9003929, user_id=6da2b52b9fcd413d8d0160032e3aeedb, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:32.771591 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ab3584-3157-4f61-a264-a8c28d3c1e1f admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:32.787357 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1043/2064] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:25:32 2022] PUT /identity/v3/projects/979a5be6112b45cfb8ab0727d9003929/users/6da2b52b9fcd413d8d0160032e3aeedb/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:25:32.791860 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222c13ad-38b4-4e28-b51d-1d4024d7f746 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:32.801939 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222c13ad-38b4-4e28-b51d-1d4024d7f746 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:32.871089 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c68e2b6-aa2f-4d59-825f-389011328e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:32.871730 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68e2b6-aa2f-4d59-825f-389011328e37 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:32.871950 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68e2b6-aa2f-4d59-825f-389011328e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:32.872205 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68e2b6-aa2f-4d59-825f-389011328e37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:32.880967 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c68e2b6-aa2f-4d59-825f-389011328e37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:32.880967 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c68e2b6-aa2f-4d59-825f-389011328e37 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:32.886193 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1022/2065] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:25:32 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:32.890863 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ab3584-3157-4f61-a264-a8c28d3c1e1f admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:32.901903 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ab3584-3157-4f61-a264-a8c28d3c1e1f admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:32.909129 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90afa31-7c39-44a7-8103-01fdb8476863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:32.910051 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90afa31-7c39-44a7-8103-01fdb8476863 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:32.910336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90afa31-7c39-44a7-8103-01fdb8476863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:32.910609 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90afa31-7c39-44a7-8103-01fdb8476863 admin admin] PATH_INFO: `/v3/projects/979a5be6112b45cfb8ab0727d9003929/users/6da2b52b9fcd413d8d0160032e3aeedb/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:32.913489 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90afa31-7c39-44a7-8103-01fdb8476863 admin admin] RBAC: Authorizing `identity:create_grant(project_id=979a5be6112b45cfb8ab0727d9003929, user_id=6da2b52b9fcd413d8d0160032e3aeedb, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:32.916867 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90afa31-7c39-44a7-8103-01fdb8476863 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:32.929009 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1044/2066] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:25:32 2022] PUT /identity/v3/projects/979a5be6112b45cfb8ab0727d9003929/users/6da2b52b9fcd413d8d0160032e3aeedb/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:32.939552 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad25c57-6afa-4ca5-92ae-a8268096c8d7 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:32.939848 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad25c57-6afa-4ca5-92ae-a8268096c8d7 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:32.940126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad25c57-6afa-4ca5-92ae-a8268096c8d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:32.960156 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:32.968241 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-bad25c57-6afa-4ca5-92ae-a8268096c8d7 None None] MFA Rules not processed for user `6da2b52b9fcd413d8d0160032e3aeedb`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:32.970053 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-bad25c57-6afa-4ca5-92ae-a8268096c8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:33.044587 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1023/2067] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:25:32 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:33.056845 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90afa31-7c39-44a7-8103-01fdb8476863 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:33.135103 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90afa31-7c39-44a7-8103-01fdb8476863 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:33.140150 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a0a19-f4ef-4609-b4ed-d0393b4cc16c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:33.140807 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a0a19-f4ef-4609-b4ed-d0393b4cc16c service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:33.141120 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a0a19-f4ef-4609-b4ed-d0393b4cc16c service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:33.141393 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a0a19-f4ef-4609-b4ed-d0393b4cc16c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:33.152190 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a0a19-f4ef-4609-b4ed-d0393b4cc16c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:33.153867 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a0a19-f4ef-4609-b4ed-d0393b4cc16c service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:33.165082 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1045/2068] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:25:33 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:25:33.221206 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad25c57-6afa-4ca5-92ae-a8268096c8d7 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:33.222634 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-bad25c57-6afa-4ca5-92ae-a8268096c8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:33.240686 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad25c57-6afa-4ca5-92ae-a8268096c8d7 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:33.251319 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ff48d-244a-4285-9621-16f4d015577d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:33.251966 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ff48d-244a-4285-9621-16f4d015577d service cinder] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:33.252255 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ff48d-244a-4285-9621-16f4d015577d service cinder] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:33.252645 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ff48d-244a-4285-9621-16f4d015577d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:33.340682 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ff48d-244a-4285-9621-16f4d015577d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:33.342374 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ff48d-244a-4285-9621-16f4d015577d service cinder] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:33.352903 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1024/2069] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:25:33 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:25:33.941363 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869a405-8f8c-4fde-92e7-3276c27a325f None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:33.941890 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869a405-8f8c-4fde-92e7-3276c27a325f None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:33.941995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869a405-8f8c-4fde-92e7-3276c27a325f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:33.962175 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-f869a405-8f8c-4fde-92e7-3276c27a325f None None] MFA Rules not processed for user `f0fbd32e5751496e8eaaf3bd03d8f833`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:33.963420 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-f869a405-8f8c-4fde-92e7-3276c27a325f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:34.010049 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:34.034830 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1046/2070] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:25:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:34.879788 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1493d3-fd78-465b-997d-660f82bcc23d None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:34.879788 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1493d3-fd78-465b-997d-660f82bcc23d None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:34.879788 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1493d3-fd78-465b-997d-660f82bcc23d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:34.906252 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-3a1493d3-fd78-465b-997d-660f82bcc23d None None] MFA Rules not processed for user `468a46d55bd342739b9cd78bcd3a77fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:34.907732 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-3a1493d3-fd78-465b-997d-660f82bcc23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:35.009302 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1025/2071] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:25:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:36.897872 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869a405-8f8c-4fde-92e7-3276c27a325f None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:37.022938 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869a405-8f8c-4fde-92e7-3276c27a325f None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:37.031566 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3c619-f1cc-451c-bcf0-599e18bc074a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:37.033494 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3c619-f1cc-451c-bcf0-599e18bc074a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:37.034679 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3c619-f1cc-451c-bcf0-599e18bc074a admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:37.035027 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb3c619-f1cc-451c-bcf0-599e18bc074a admin admin] PATH_INFO: `/v3/users/bb82e297d6e5438898d5e5fad1756e79` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:37.036981 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb3c619-f1cc-451c-bcf0-599e18bc074a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb82e297d6e5438898d5e5fad1756e79)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:37.039237 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb3c619-f1cc-451c-bcf0-599e18bc074a admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:37.152528 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-2bb3c619-f1cc-451c-bcf0-599e18bc074a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb82e297d6e5438898d5e5fad1756e79'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:37.153622 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-2bb3c619-f1cc-451c-bcf0-599e18bc074a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb82e297d6e5438898d5e5fad1756e79'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:37.166694 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-2bb3c619-f1cc-451c-bcf0-599e18bc074a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb82e297d6e5438898d5e5fad1756e79'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:37.171153 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-2bb3c619-f1cc-451c-bcf0-599e18bc074a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb82e297d6e5438898d5e5fad1756e79'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:37.192060 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-2bb3c619-f1cc-451c-bcf0-599e18bc074a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb82e297d6e5438898d5e5fad1756e79'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:37.215004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1047/2072] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:25:36 2022] DELETE /identity/v3/users/bb82e297d6e5438898d5e5fad1756e79 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:37.219772 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0f4f6-4e16-4a6b-8529-873ce516aa2a None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:37.220261 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0f4f6-4e16-4a6b-8529-873ce516aa2a None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:37.220339 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa0f4f6-4e16-4a6b-8529-873ce516aa2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:37.240888 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-dfa0f4f6-4e16-4a6b-8529-873ce516aa2a None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:37.242826 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-dfa0f4f6-4e16-4a6b-8529-873ce516aa2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:37.286392 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:37.314904 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1026/2073] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:25:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:37.326875 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3c619-f1cc-451c-bcf0-599e18bc074a admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:37.328378 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-2bb3c619-f1cc-451c-bcf0-599e18bc074a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:37.393841 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb3c619-f1cc-451c-bcf0-599e18bc074a admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:37.397719 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10face75-7177-4afa-b726-442d8f536c7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:37.398385 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10face75-7177-4afa-b726-442d8f536c7e service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:37.398648 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10face75-7177-4afa-b726-442d8f536c7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:37.398918 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10face75-7177-4afa-b726-442d8f536c7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:37.412771 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10face75-7177-4afa-b726-442d8f536c7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:37.414271 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10face75-7177-4afa-b726-442d8f536c7e service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:37.425012 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1048/2074] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:25:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:25:37.739781 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0f4f6-4e16-4a6b-8529-873ce516aa2a None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:37.741386 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-dfa0f4f6-4e16-4a6b-8529-873ce516aa2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:37.756403 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa0f4f6-4e16-4a6b-8529-873ce516aa2a None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:37.762795 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe41b53-3ffe-4c52-b2f0-c5b1650f7e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:37.764027 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe41b53-3ffe-4c52-b2f0-c5b1650f7e47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:37.764308 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe41b53-3ffe-4c52-b2f0-c5b1650f7e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:37.764644 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe41b53-3ffe-4c52-b2f0-c5b1650f7e47 admin admin] PATH_INFO: `/v3/projects/750d1d9f4e714ac7a31641006ecfd89b` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:37.765804 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe41b53-3ffe-4c52-b2f0-c5b1650f7e47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=750d1d9f4e714ac7a31641006ecfd89b)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:37.767739 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe41b53-3ffe-4c52-b2f0-c5b1650f7e47 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:37.820450 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-abe41b53-3ffe-4c52-b2f0-c5b1650f7e47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '750d1d9f4e714ac7a31641006ecfd89b', 'request_id': 'req-abe41b53-3ffe-4c52-b2f0-c5b1650f7e47'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:37.830801 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-abe41b53-3ffe-4c52-b2f0-c5b1650f7e47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '750d1d9f4e714ac7a31641006ecfd89b', 'request_id': 'req-abe41b53-3ffe-4c52-b2f0-c5b1650f7e47'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:37.840132 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-abe41b53-3ffe-4c52-b2f0-c5b1650f7e47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '750d1d9f4e714ac7a31641006ecfd89b', 'request_id': 'req-abe41b53-3ffe-4c52-b2f0-c5b1650f7e47'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:37.856694 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-abe41b53-3ffe-4c52-b2f0-c5b1650f7e47 admin admin] The token cache is being invalidate because project 750d1d9f4e714ac7a31641006ecfd89b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:25:37.857010 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-abe41b53-3ffe-4c52-b2f0-c5b1650f7e47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:37.857676 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-abe41b53-3ffe-4c52-b2f0-c5b1650f7e47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:37.859316 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1027/2075] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:25:37 2022] DELETE /identity/v3/projects/750d1d9f4e714ac7a31641006ecfd89b => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:37.863715 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10face75-7177-4afa-b726-442d8f536c7e service neutron] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:37.874718 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10face75-7177-4afa-b726-442d8f536c7e service neutron] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:37.878915 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:37.941504 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7bb544-a4a7-4d24-b7e3-af7c935cb1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:37.942390 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bb544-a4a7-4d24-b7e3-af7c935cb1d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:37.942693 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bb544-a4a7-4d24-b7e3-af7c935cb1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:37.942967 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bb544-a4a7-4d24-b7e3-af7c935cb1d6 admin admin] PATH_INFO: `/v3/users/6da2b52b9fcd413d8d0160032e3aeedb` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:37.944219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7bb544-a4a7-4d24-b7e3-af7c935cb1d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6da2b52b9fcd413d8d0160032e3aeedb)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:37.946573 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7bb544-a4a7-4d24-b7e3-af7c935cb1d6 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:38.005632 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-4b7bb544-a4a7-4d24-b7e3-af7c935cb1d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6da2b52b9fcd413d8d0160032e3aeedb'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:38.006444 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-4b7bb544-a4a7-4d24-b7e3-af7c935cb1d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6da2b52b9fcd413d8d0160032e3aeedb'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:38.016223 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-4b7bb544-a4a7-4d24-b7e3-af7c935cb1d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6da2b52b9fcd413d8d0160032e3aeedb'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:38.016849 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-4b7bb544-a4a7-4d24-b7e3-af7c935cb1d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6da2b52b9fcd413d8d0160032e3aeedb'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:38.029778 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-4b7bb544-a4a7-4d24-b7e3-af7c935cb1d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6da2b52b9fcd413d8d0160032e3aeedb'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:38.043327 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1049/2076] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:25:37 2022] DELETE /identity/v3/users/6da2b52b9fcd413d8d0160032e3aeedb => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:38.492632 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe41b53-3ffe-4c52-b2f0-c5b1650f7e47 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:38.494249 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-abe41b53-3ffe-4c52-b2f0-c5b1650f7e47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:38.518892 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe41b53-3ffe-4c52-b2f0-c5b1650f7e47 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:38.585803 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5643cf6a-6446-4744-b3ad-bf5ae91a9fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:38.586735 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643cf6a-6446-4744-b3ad-bf5ae91a9fc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:38.587201 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643cf6a-6446-4744-b3ad-bf5ae91a9fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:38.587484 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5643cf6a-6446-4744-b3ad-bf5ae91a9fc1 admin admin] PATH_INFO: `/v3/projects/979a5be6112b45cfb8ab0727d9003929` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:38.597587 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5643cf6a-6446-4744-b3ad-bf5ae91a9fc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=979a5be6112b45cfb8ab0727d9003929)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:38.597587 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5643cf6a-6446-4744-b3ad-bf5ae91a9fc1 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:38.648821 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-5643cf6a-6446-4744-b3ad-bf5ae91a9fc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '979a5be6112b45cfb8ab0727d9003929', 'request_id': 'req-5643cf6a-6446-4744-b3ad-bf5ae91a9fc1'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:38.655351 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-5643cf6a-6446-4744-b3ad-bf5ae91a9fc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '979a5be6112b45cfb8ab0727d9003929', 'request_id': 'req-5643cf6a-6446-4744-b3ad-bf5ae91a9fc1'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:38.661071 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-5643cf6a-6446-4744-b3ad-bf5ae91a9fc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '979a5be6112b45cfb8ab0727d9003929', 'request_id': 'req-5643cf6a-6446-4744-b3ad-bf5ae91a9fc1'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:38.663723 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:38.676990 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-5643cf6a-6446-4744-b3ad-bf5ae91a9fc1 admin admin] The token cache is being invalidate because project 979a5be6112b45cfb8ab0727d9003929 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:25:38.677272 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-5643cf6a-6446-4744-b3ad-bf5ae91a9fc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:38.678329 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-5643cf6a-6446-4744-b3ad-bf5ae91a9fc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:38.679933 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1028/2077] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:25:38 2022] DELETE /identity/v3/projects/979a5be6112b45cfb8ab0727d9003929 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:38.685086 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6deaa-722a-4042-9d08-fe2e994355e5 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:38.685438 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6deaa-722a-4042-9d08-fe2e994355e5 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:38.685784 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6deaa-722a-4042-9d08-fe2e994355e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:38.703724 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-42f6deaa-722a-4042-9d08-fe2e994355e5 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:38.710050 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-42f6deaa-722a-4042-9d08-fe2e994355e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:38.771183 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1050/2078] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:25:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:25:38.779443 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772457a2-7389-49c2-b2dd-16a7afc272bd None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:38.779759 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772457a2-7389-49c2-b2dd-16a7afc272bd None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:38.780040 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772457a2-7389-49c2-b2dd-16a7afc272bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:38.799039 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-772457a2-7389-49c2-b2dd-16a7afc272bd None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:38.800238 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-772457a2-7389-49c2-b2dd-16a7afc272bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:38.809706 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1029/2079] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:25:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:38.814255 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f6deaa-722a-4042-9d08-fe2e994355e5 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:38.824630 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f6deaa-722a-4042-9d08-fe2e994355e5 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:38.830070 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bda7fe-8775-4fb0-974d-57b662bd5327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:38.830722 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bda7fe-8775-4fb0-974d-57b662bd5327 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:38.830996 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bda7fe-8775-4fb0-974d-57b662bd5327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:38.831265 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bda7fe-8775-4fb0-974d-57b662bd5327 admin admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:38.832088 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bda7fe-8775-4fb0-974d-57b662bd5327 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:25:38.832384 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bda7fe-8775-4fb0-974d-57b662bd5327 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:38.834159 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bda7fe-8775-4fb0-974d-57b662bd5327 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:38.846540 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1051/2080] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:25:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:25:38.851323 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772457a2-7389-49c2-b2dd-16a7afc272bd None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:38.861370 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772457a2-7389-49c2-b2dd-16a7afc272bd None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:38.866491 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9538f-35dc-4026-9abb-08040d777d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:38.867177 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9538f-35dc-4026-9abb-08040d777d22 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:38.867449 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9538f-35dc-4026-9abb-08040d777d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:38.867716 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9538f-35dc-4026-9abb-08040d777d22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:38.868398 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9538f-35dc-4026-9abb-08040d777d22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:38.870354 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9538f-35dc-4026-9abb-08040d777d22 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:38.898720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1030/2081] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:25:38 2022] POST /identity/v3/projects => generated 389 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:25:38.903005 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bda7fe-8775-4fb0-974d-57b662bd5327 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:38.914134 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bda7fe-8775-4fb0-974d-57b662bd5327 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:38.923761 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:38.975830 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0d0ea-cb12-4e1a-a5b4-5b265a1955e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:38.976546 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0d0ea-cb12-4e1a-a5b4-5b265a1955e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:38.976835 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0d0ea-cb12-4e1a-a5b4-5b265a1955e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:38.977137 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0d0ea-cb12-4e1a-a5b4-5b265a1955e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:38.977816 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f0d0ea-cb12-4e1a-a5b4-5b265a1955e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:38.979760 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f0d0ea-cb12-4e1a-a5b4-5b265a1955e5 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:39.001333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1052/2082] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:25:38 2022] POST /identity/v3/users => generated 295 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:25:39.005590 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9538f-35dc-4026-9abb-08040d777d22 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:39.016715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9538f-35dc-4026-9abb-08040d777d22 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:39.022343 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c442b427-ab32-4406-aff4-dfc1678f4ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:39.023011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442b427-ab32-4406-aff4-dfc1678f4ba9 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:39.023291 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442b427-ab32-4406-aff4-dfc1678f4ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:39.023562 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442b427-ab32-4406-aff4-dfc1678f4ba9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:39.024303 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c442b427-ab32-4406-aff4-dfc1678f4ba9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:39.025966 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c442b427-ab32-4406-aff4-dfc1678f4ba9 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:39.037276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1031/2083] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:39.041478 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0d0ea-cb12-4e1a-a5b4-5b265a1955e5 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:39.052097 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0d0ea-cb12-4e1a-a5b4-5b265a1955e5 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:39.057982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783eae6-6aa8-45e6-8cdd-d443d6c75d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:39.058671 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783eae6-6aa8-45e6-8cdd-d443d6c75d2f admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:39.058947 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783eae6-6aa8-45e6-8cdd-d443d6c75d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:39.059216 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783eae6-6aa8-45e6-8cdd-d443d6c75d2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:39.060288 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9783eae6-6aa8-45e6-8cdd-d443d6c75d2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:39.062386 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9783eae6-6aa8-45e6-8cdd-d443d6c75d2f admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:39.072788 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1053/2084] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:39.077195 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c442b427-ab32-4406-aff4-dfc1678f4ba9 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:39.088171 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c442b427-ab32-4406-aff4-dfc1678f4ba9 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:39.094490 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4001c8f-9058-47f3-9588-4a6d4471f4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:39.095532 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4001c8f-9058-47f3-9588-4a6d4471f4a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:39.095870 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4001c8f-9058-47f3-9588-4a6d4471f4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:39.096285 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4001c8f-9058-47f3-9588-4a6d4471f4a8 admin admin] PATH_INFO: `/v3/projects/48ea3b227a5c45e28cb31d639dab3754/users/b1922e80ad774f85b489dc9c2ab8c678/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:39.115226 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4001c8f-9058-47f3-9588-4a6d4471f4a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48ea3b227a5c45e28cb31d639dab3754, user_id=b1922e80ad774f85b489dc9c2ab8c678, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:39.118074 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4001c8f-9058-47f3-9588-4a6d4471f4a8 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:39.132249 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1032/2085] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:25:39 2022] PUT /identity/v3/projects/48ea3b227a5c45e28cb31d639dab3754/users/b1922e80ad774f85b489dc9c2ab8c678/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:39.145086 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c0076-7cd0-4520-a066-e0bb835981c7 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:39.145376 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c0076-7cd0-4520-a066-e0bb835981c7 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:39.145648 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c0076-7cd0-4520-a066-e0bb835981c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:39.176540 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-201c0076-7cd0-4520-a066-e0bb835981c7 None None] MFA Rules not processed for user `b1922e80ad774f85b489dc9c2ab8c678`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:39.177794 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-201c0076-7cd0-4520-a066-e0bb835981c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:39.253449 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1054/2086] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:25:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:39.258099 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4001c8f-9058-47f3-9588-4a6d4471f4a8 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:39.332391 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4001c8f-9058-47f3-9588-4a6d4471f4a8 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:39.337769 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379a36c-09d8-4f46-aeae-20e76e2f50be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:39.338387 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379a36c-09d8-4f46-aeae-20e76e2f50be admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:39.338668 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379a36c-09d8-4f46-aeae-20e76e2f50be admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:39.339048 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379a36c-09d8-4f46-aeae-20e76e2f50be admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:39.339766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4379a36c-09d8-4f46-aeae-20e76e2f50be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:39.341755 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4379a36c-09d8-4f46-aeae-20e76e2f50be admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:39.374976 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1033/2087] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:25:39 2022] POST /identity/v3/projects => generated 389 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:25:39.376593 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201c0076-7cd0-4520-a066-e0bb835981c7 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:39.389855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201c0076-7cd0-4520-a066-e0bb835981c7 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:39.459680 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b996749d-253b-4a6a-bbd4-fa5a42c316c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:39.460371 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996749d-253b-4a6a-bbd4-fa5a42c316c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:39.460655 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996749d-253b-4a6a-bbd4-fa5a42c316c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:39.461254 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996749d-253b-4a6a-bbd4-fa5a42c316c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:39.461969 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b996749d-253b-4a6a-bbd4-fa5a42c316c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:39.463928 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b996749d-253b-4a6a-bbd4-fa5a42c316c7 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:39.485388 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1055/2088] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:25:39 2022] POST /identity/v3/users => generated 301 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:25:39.489902 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379a36c-09d8-4f46-aeae-20e76e2f50be admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:39.502907 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379a36c-09d8-4f46-aeae-20e76e2f50be admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:39.511383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7946144-21bb-470d-9809-00638f3f4514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:39.513524 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7946144-21bb-470d-9809-00638f3f4514 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:39.514155 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7946144-21bb-470d-9809-00638f3f4514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:39.515465 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7946144-21bb-470d-9809-00638f3f4514 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:39.516863 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7946144-21bb-470d-9809-00638f3f4514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:39.518598 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7946144-21bb-470d-9809-00638f3f4514 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:39.537579 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1034/2089] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:39.539010 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b996749d-253b-4a6a-bbd4-fa5a42c316c7 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:39.550091 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b996749d-253b-4a6a-bbd4-fa5a42c316c7 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:39.556512 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf294f-d5d5-4a78-877f-3a2e1952ee18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:39.557502 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf294f-d5d5-4a78-877f-3a2e1952ee18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:39.557777 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf294f-d5d5-4a78-877f-3a2e1952ee18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:39.558096 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf294f-d5d5-4a78-877f-3a2e1952ee18 admin admin] PATH_INFO: `/v3/projects/768dc2a5e1f44431be1b10e8d031e625/users/61708d8bef1a41d385811634c10919c6/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:39.573149 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:39.582146 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccf294f-d5d5-4a78-877f-3a2e1952ee18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=768dc2a5e1f44431be1b10e8d031e625, user_id=61708d8bef1a41d385811634c10919c6, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:39.585151 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccf294f-d5d5-4a78-877f-3a2e1952ee18 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:39.611038 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1056/2090] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:25:39 2022] PUT /identity/v3/projects/768dc2a5e1f44431be1b10e8d031e625/users/61708d8bef1a41d385811634c10919c6/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:39.612058 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7946144-21bb-470d-9809-00638f3f4514 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:39.623515 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7946144-21bb-470d-9809-00638f3f4514 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:39.693735 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d694a77-70c3-4923-be48-26f28a728c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:39.694415 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d694a77-70c3-4923-be48-26f28a728c68 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:39.694698 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d694a77-70c3-4923-be48-26f28a728c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:39.695101 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d694a77-70c3-4923-be48-26f28a728c68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:39.695839 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d694a77-70c3-4923-be48-26f28a728c68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:39.697464 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d694a77-70c3-4923-be48-26f28a728c68 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:39.703281 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:39.709165 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1035/2091] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:25:39 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:25:39.712748 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf294f-d5d5-4a78-877f-3a2e1952ee18 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:39.724929 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf294f-d5d5-4a78-877f-3a2e1952ee18 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:39.729910 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7d0de-a268-4664-9eb2-467a47a38388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:39.730751 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7d0de-a268-4664-9eb2-467a47a38388 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:39.731027 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7d0de-a268-4664-9eb2-467a47a38388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:39.731304 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe7d0de-a268-4664-9eb2-467a47a38388 admin admin] PATH_INFO: `/v3/projects/768dc2a5e1f44431be1b10e8d031e625/users/61708d8bef1a41d385811634c10919c6/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:39.733333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe7d0de-a268-4664-9eb2-467a47a38388 admin admin] RBAC: Authorizing `identity:create_grant(project_id=768dc2a5e1f44431be1b10e8d031e625, user_id=61708d8bef1a41d385811634c10919c6, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:39.736624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe7d0de-a268-4664-9eb2-467a47a38388 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:39.748890 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1057/2092] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:25:39 2022] PUT /identity/v3/projects/768dc2a5e1f44431be1b10e8d031e625/users/61708d8bef1a41d385811634c10919c6/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:39.765725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b1fd2e-e222-45a2-861b-550a926b9d5f None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:39.766111 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b1fd2e-e222-45a2-861b-550a926b9d5f None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:39.766454 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b1fd2e-e222-45a2-861b-550a926b9d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:39.793753 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-10b1fd2e-e222-45a2-861b-550a926b9d5f None None] MFA Rules not processed for user `61708d8bef1a41d385811634c10919c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:39.795097 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-10b1fd2e-e222-45a2-861b-550a926b9d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:39.867031 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1036/2093] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:25:39 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:39.878700 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7d0de-a268-4664-9eb2-467a47a38388 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:39.880880 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-cfe7d0de-a268-4664-9eb2-467a47a38388 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:39.973335 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe7d0de-a268-4664-9eb2-467a47a38388 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:39.977446 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a77171-1c8a-4980-9461-2e6adfef15e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:39.978059 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a77171-1c8a-4980-9461-2e6adfef15e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:39.978354 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a77171-1c8a-4980-9461-2e6adfef15e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:39.978627 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a77171-1c8a-4980-9461-2e6adfef15e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:39.988732 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a77171-1c8a-4980-9461-2e6adfef15e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:39.990144 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a77171-1c8a-4980-9461-2e6adfef15e7 service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:40.000035 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1058/2094] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:25:39 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:25:40.141829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b1fd2e-e222-45a2-861b-550a926b9d5f None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:40.205877 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b1fd2e-e222-45a2-861b-550a926b9d5f None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:40.210802 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d2b36-dc81-45c4-b9d5-8524dbdb9c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:40.211683 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d2b36-dc81-45c4-b9d5-8524dbdb9c82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:40.211965 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d2b36-dc81-45c4-b9d5-8524dbdb9c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:40.212241 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d2b36-dc81-45c4-b9d5-8524dbdb9c82 admin admin] PATH_INFO: `/v3/users/b1922e80ad774f85b489dc9c2ab8c678` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:40.213540 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0d2b36-dc81-45c4-b9d5-8524dbdb9c82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1922e80ad774f85b489dc9c2ab8c678)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:40.215509 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0d2b36-dc81-45c4-b9d5-8524dbdb9c82 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:40.277315 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-cc0d2b36-dc81-45c4-b9d5-8524dbdb9c82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1922e80ad774f85b489dc9c2ab8c678'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:40.290313 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-cc0d2b36-dc81-45c4-b9d5-8524dbdb9c82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1922e80ad774f85b489dc9c2ab8c678'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:40.304268 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-cc0d2b36-dc81-45c4-b9d5-8524dbdb9c82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1922e80ad774f85b489dc9c2ab8c678'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:40.314059 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-cc0d2b36-dc81-45c4-b9d5-8524dbdb9c82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1922e80ad774f85b489dc9c2ab8c678'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:40.314960 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-cc0d2b36-dc81-45c4-b9d5-8524dbdb9c82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1922e80ad774f85b489dc9c2ab8c678'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:40.324674 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1037/2095] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:25:40 2022] DELETE /identity/v3/users/b1922e80ad774f85b489dc9c2ab8c678 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:40.330287 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9f1df-ce7f-4ca5-a882-f9736c20e730 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:40.330591 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9f1df-ce7f-4ca5-a882-f9736c20e730 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:40.330880 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9f1df-ce7f-4ca5-a882-f9736c20e730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:40.349332 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-31d9f1df-ce7f-4ca5-a882-f9736c20e730 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:40.350527 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-31d9f1df-ce7f-4ca5-a882-f9736c20e730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:40.420226 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1059/2096] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:25:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:40.432576 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d2b36-dc81-45c4-b9d5-8524dbdb9c82 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:40.434126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-cc0d2b36-dc81-45c4-b9d5-8524dbdb9c82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:40.442985 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:40.503448 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d2b36-dc81-45c4-b9d5-8524dbdb9c82 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:40.514698 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea7556b-861b-4891-a30b-7c66174640e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:40.515532 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7556b-861b-4891-a30b-7c66174640e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:40.515802 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7556b-861b-4891-a30b-7c66174640e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:40.516071 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7556b-861b-4891-a30b-7c66174640e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:40.525496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea7556b-861b-4891-a30b-7c66174640e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:40.526920 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea7556b-861b-4891-a30b-7c66174640e6 service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:40.536371 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1038/2097] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:25:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:25:40.956993 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9f1df-ce7f-4ca5-a882-f9736c20e730 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:40.958527 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-31d9f1df-ce7f-4ca5-a882-f9736c20e730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:40.972049 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9f1df-ce7f-4ca5-a882-f9736c20e730 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:40.978606 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b43214-c20c-49fb-85d6-a728d6e32d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:40.979670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b43214-c20c-49fb-85d6-a728d6e32d06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:40.979935 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b43214-c20c-49fb-85d6-a728d6e32d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:40.980492 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b43214-c20c-49fb-85d6-a728d6e32d06 admin admin] PATH_INFO: `/v3/projects/48ea3b227a5c45e28cb31d639dab3754` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:40.981911 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b43214-c20c-49fb-85d6-a728d6e32d06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48ea3b227a5c45e28cb31d639dab3754)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:40.985098 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b43214-c20c-49fb-85d6-a728d6e32d06 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:41.049192 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-89b43214-c20c-49fb-85d6-a728d6e32d06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48ea3b227a5c45e28cb31d639dab3754', 'request_id': 'req-89b43214-c20c-49fb-85d6-a728d6e32d06'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:41.051380 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:41.056162 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-89b43214-c20c-49fb-85d6-a728d6e32d06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48ea3b227a5c45e28cb31d639dab3754', 'request_id': 'req-89b43214-c20c-49fb-85d6-a728d6e32d06'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:41.065381 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-89b43214-c20c-49fb-85d6-a728d6e32d06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48ea3b227a5c45e28cb31d639dab3754', 'request_id': 'req-89b43214-c20c-49fb-85d6-a728d6e32d06'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:41.076473 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-89b43214-c20c-49fb-85d6-a728d6e32d06 admin admin] The token cache is being invalidate because project 48ea3b227a5c45e28cb31d639dab3754 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:25:41.076755 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-89b43214-c20c-49fb-85d6-a728d6e32d06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:41.077468 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-89b43214-c20c-49fb-85d6-a728d6e32d06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:41.079222 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1060/2098] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:25:40 2022] DELETE /identity/v3/projects/48ea3b227a5c45e28cb31d639dab3754 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:41.083419 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea7556b-861b-4891-a30b-7c66174640e6 service neutron] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:41.094490 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea7556b-861b-4891-a30b-7c66174640e6 service neutron] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:41.150196 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed4c74e-1e4f-452f-8d58-3e4ffb66b56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:41.150988 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4c74e-1e4f-452f-8d58-3e4ffb66b56e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:41.151265 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4c74e-1e4f-452f-8d58-3e4ffb66b56e admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:41.151553 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed4c74e-1e4f-452f-8d58-3e4ffb66b56e admin admin] PATH_INFO: `/v3/users/61708d8bef1a41d385811634c10919c6` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:41.152703 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed4c74e-1e4f-452f-8d58-3e4ffb66b56e admin admin] RBAC: Authorizing `identity:delete_user(user_id=61708d8bef1a41d385811634c10919c6)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:41.154657 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed4c74e-1e4f-452f-8d58-3e4ffb66b56e admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:41.205022 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-5ed4c74e-1e4f-452f-8d58-3e4ffb66b56e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61708d8bef1a41d385811634c10919c6'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:41.216923 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-5ed4c74e-1e4f-452f-8d58-3e4ffb66b56e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61708d8bef1a41d385811634c10919c6'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:41.227013 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-5ed4c74e-1e4f-452f-8d58-3e4ffb66b56e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61708d8bef1a41d385811634c10919c6'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:41.235479 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-5ed4c74e-1e4f-452f-8d58-3e4ffb66b56e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61708d8bef1a41d385811634c10919c6'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:41.236216 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-5ed4c74e-1e4f-452f-8d58-3e4ffb66b56e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61708d8bef1a41d385811634c10919c6'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:41.242975 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1039/2099] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:25:41 2022] DELETE /identity/v3/users/61708d8bef1a41d385811634c10919c6 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:41.524569 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b43214-c20c-49fb-85d6-a728d6e32d06 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:41.527624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-89b43214-c20c-49fb-85d6-a728d6e32d06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:41.544970 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b43214-c20c-49fb-85d6-a728d6e32d06 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:41.607523 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0616e4-839c-4607-9153-851120e64701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:41.608435 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0616e4-839c-4607-9153-851120e64701 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:41.608821 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0616e4-839c-4607-9153-851120e64701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:41.609211 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0616e4-839c-4607-9153-851120e64701 admin admin] PATH_INFO: `/v3/projects/768dc2a5e1f44431be1b10e8d031e625` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:41.610401 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0616e4-839c-4607-9153-851120e64701 admin admin] RBAC: Authorizing `identity:delete_project(project_id=768dc2a5e1f44431be1b10e8d031e625)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:41.612429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0616e4-839c-4607-9153-851120e64701 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:41.660923 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-5c0616e4-839c-4607-9153-851120e64701 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '768dc2a5e1f44431be1b10e8d031e625', 'request_id': 'req-5c0616e4-839c-4607-9153-851120e64701'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:41.667249 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-5c0616e4-839c-4607-9153-851120e64701 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '768dc2a5e1f44431be1b10e8d031e625', 'request_id': 'req-5c0616e4-839c-4607-9153-851120e64701'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:41.675882 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-5c0616e4-839c-4607-9153-851120e64701 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '768dc2a5e1f44431be1b10e8d031e625', 'request_id': 'req-5c0616e4-839c-4607-9153-851120e64701'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:41.686199 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-5c0616e4-839c-4607-9153-851120e64701 admin admin] The token cache is being invalidate because project 768dc2a5e1f44431be1b10e8d031e625 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:25:41.686511 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-5c0616e4-839c-4607-9153-851120e64701 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:41.687254 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-5c0616e4-839c-4607-9153-851120e64701 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:41.689140 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1061/2100] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:25:41 2022] DELETE /identity/v3/projects/768dc2a5e1f44431be1b10e8d031e625 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:41.694303 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fefb671-859a-4d80-996f-c45cf3083773 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:41.695238 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fefb671-859a-4d80-996f-c45cf3083773 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:41.695238 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fefb671-859a-4d80-996f-c45cf3083773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:41.714044 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-9fefb671-859a-4d80-996f-c45cf3083773 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:41.715251 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-9fefb671-859a-4d80-996f-c45cf3083773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:41.735846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:41.780434 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1040/2101] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:25:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:41.789555 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f91bab-4068-4c29-b575-74382317bcc0 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:41.789892 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f91bab-4068-4c29-b575-74382317bcc0 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:41.790168 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f91bab-4068-4c29-b575-74382317bcc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:41.809091 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-70f91bab-4068-4c29-b575-74382317bcc0 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:41.810430 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-70f91bab-4068-4c29-b575-74382317bcc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:41.822734 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1062/2102] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:25:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:41.823929 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fefb671-859a-4d80-996f-c45cf3083773 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:41.840334 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fefb671-859a-4d80-996f-c45cf3083773 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:41.850466 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029287c0-0c2e-447c-bb7d-d36c0fc56e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:41.851115 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029287c0-0c2e-447c-bb7d-d36c0fc56e9d admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:41.851389 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029287c0-0c2e-447c-bb7d-d36c0fc56e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:41.851653 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029287c0-0c2e-447c-bb7d-d36c0fc56e9d admin admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:41.852430 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029287c0-0c2e-447c-bb7d-d36c0fc56e9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:25:41.852679 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029287c0-0c2e-447c-bb7d-d36c0fc56e9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:41.854402 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029287c0-0c2e-447c-bb7d-d36c0fc56e9d admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:41.867724 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1041/2103] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:25:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:25:41.872236 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f91bab-4068-4c29-b575-74382317bcc0 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:41.884084 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f91bab-4068-4c29-b575-74382317bcc0 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:41.890160 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4028e9ce-fe62-4ef7-931c-ddd9eb157d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:41.890883 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028e9ce-fe62-4ef7-931c-ddd9eb157d8e admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:41.891209 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028e9ce-fe62-4ef7-931c-ddd9eb157d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:41.891439 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028e9ce-fe62-4ef7-931c-ddd9eb157d8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:41.892260 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4028e9ce-fe62-4ef7-931c-ddd9eb157d8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:41.894252 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4028e9ce-fe62-4ef7-931c-ddd9eb157d8e admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:41.933946 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1063/2104] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:25:41 2022] POST /identity/v3/projects => generated 383 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:25:41.937885 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029287c0-0c2e-447c-bb7d-d36c0fc56e9d admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:41.948344 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029287c0-0c2e-447c-bb7d-d36c0fc56e9d admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:42.013894 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fafc6f-20af-4a9c-84df-6631fa39367b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:42.014596 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fafc6f-20af-4a9c-84df-6631fa39367b admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:42.014926 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fafc6f-20af-4a9c-84df-6631fa39367b admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:42.015386 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fafc6f-20af-4a9c-84df-6631fa39367b admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:42.016198 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fafc6f-20af-4a9c-84df-6631fa39367b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:42.018155 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fafc6f-20af-4a9c-84df-6631fa39367b admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:42.040803 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1042/2105] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:25:41 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:25:42.044969 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4028e9ce-fe62-4ef7-931c-ddd9eb157d8e admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:42.055908 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4028e9ce-fe62-4ef7-931c-ddd9eb157d8e admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:42.060935 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee22ff13-0302-417f-be45-7eb23e5fb957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:42.061630 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22ff13-0302-417f-be45-7eb23e5fb957 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:42.062035 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22ff13-0302-417f-be45-7eb23e5fb957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:42.062320 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee22ff13-0302-417f-be45-7eb23e5fb957 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:42.063083 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee22ff13-0302-417f-be45-7eb23e5fb957 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:42.064798 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee22ff13-0302-417f-be45-7eb23e5fb957 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:42.076671 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1064/2106] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:25:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:42.081467 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fafc6f-20af-4a9c-84df-6631fa39367b admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:42.091854 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fafc6f-20af-4a9c-84df-6631fa39367b admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:42.098643 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b3ee1-45a8-4d16-9965-d0f1e8b62519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:42.099970 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b3ee1-45a8-4d16-9965-d0f1e8b62519 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:42.099970 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b3ee1-45a8-4d16-9965-d0f1e8b62519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:42.099970 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b3ee1-45a8-4d16-9965-d0f1e8b62519 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:42.100793 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1b3ee1-45a8-4d16-9965-d0f1e8b62519 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:42.102645 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1b3ee1-45a8-4d16-9965-d0f1e8b62519 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:42.113866 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1043/2107] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:25:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:42.118163 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee22ff13-0302-417f-be45-7eb23e5fb957 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:42.129835 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee22ff13-0302-417f-be45-7eb23e5fb957 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:42.135141 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602adaf-5669-4713-8716-30bce1e5f460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:42.136143 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602adaf-5669-4713-8716-30bce1e5f460 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:42.136496 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602adaf-5669-4713-8716-30bce1e5f460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:42.136867 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602adaf-5669-4713-8716-30bce1e5f460 admin admin] PATH_INFO: `/v3/projects/f2e524619d7f49478237e5bb41115c11/users/4fe482c1db9e4fabb0b0f6c9912d5c21/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:42.153729 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1602adaf-5669-4713-8716-30bce1e5f460 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2e524619d7f49478237e5bb41115c11, user_id=4fe482c1db9e4fabb0b0f6c9912d5c21, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:42.156746 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1602adaf-5669-4713-8716-30bce1e5f460 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:42.173001 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1065/2108] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:25:42 2022] PUT /identity/v3/projects/f2e524619d7f49478237e5bb41115c11/users/4fe482c1db9e4fabb0b0f6c9912d5c21/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:42.183491 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8bf1b-dcd8-4a43-9ad9-d135becf055b None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:42.183803 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8bf1b-dcd8-4a43-9ad9-d135becf055b None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:42.184072 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8bf1b-dcd8-4a43-9ad9-d135becf055b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:42.214169 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-71b8bf1b-dcd8-4a43-9ad9-d135becf055b None None] MFA Rules not processed for user `4fe482c1db9e4fabb0b0f6c9912d5c21`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:42.215389 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-71b8bf1b-dcd8-4a43-9ad9-d135becf055b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:42.285953 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1044/2109] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:25:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:42.290544 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602adaf-5669-4713-8716-30bce1e5f460 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:42.300219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:42.357830 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602adaf-5669-4713-8716-30bce1e5f460 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:42.363814 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce41da4c-0c3f-4660-96bc-8dcdd7423286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:42.364493 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce41da4c-0c3f-4660-96bc-8dcdd7423286 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:42.364770 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce41da4c-0c3f-4660-96bc-8dcdd7423286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:42.365072 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce41da4c-0c3f-4660-96bc-8dcdd7423286 admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:42.365775 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce41da4c-0c3f-4660-96bc-8dcdd7423286 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:42.367727 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce41da4c-0c3f-4660-96bc-8dcdd7423286 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:42.396758 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1066/2110] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:25:42 2022] POST /identity/v3/projects => generated 381 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:25:42.401910 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b8bf1b-dcd8-4a43-9ad9-d135becf055b None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:42.415026 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b8bf1b-dcd8-4a43-9ad9-d135becf055b None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:42.436821 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:42.478328 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabff727-23d5-4da8-b4ed-faf12ca516cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:42.479021 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabff727-23d5-4da8-b4ed-faf12ca516cd admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:42.479296 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabff727-23d5-4da8-b4ed-faf12ca516cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:42.479573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabff727-23d5-4da8-b4ed-faf12ca516cd admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:42.480260 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabff727-23d5-4da8-b4ed-faf12ca516cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:42.482236 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabff727-23d5-4da8-b4ed-faf12ca516cd admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:42.506951 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1045/2111] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:25:42 2022] POST /identity/v3/users => generated 297 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:25:42.511071 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce41da4c-0c3f-4660-96bc-8dcdd7423286 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:42.522441 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce41da4c-0c3f-4660-96bc-8dcdd7423286 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:42.527101 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fa8870-936e-42dc-b131-84740dc810bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:42.527768 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa8870-936e-42dc-b131-84740dc810bf admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:42.528026 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa8870-936e-42dc-b131-84740dc810bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:42.528298 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa8870-936e-42dc-b131-84740dc810bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:42.529019 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fa8870-936e-42dc-b131-84740dc810bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:42.531228 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fa8870-936e-42dc-b131-84740dc810bf admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:42.542127 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1067/2112] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:25:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:42.547091 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabff727-23d5-4da8-b4ed-faf12ca516cd admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:42.557056 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabff727-23d5-4da8-b4ed-faf12ca516cd admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:42.562273 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e51ee0f-6314-4f17-a0d1-0ff2391d7f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:42.563175 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e51ee0f-6314-4f17-a0d1-0ff2391d7f79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:42.563455 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e51ee0f-6314-4f17-a0d1-0ff2391d7f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:42.563726 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e51ee0f-6314-4f17-a0d1-0ff2391d7f79 admin admin] PATH_INFO: `/v3/projects/7a67727c9142485fa2d09a36200f90ed/users/12041f02fb4146a2933846642764e8e9/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:42.579367 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e51ee0f-6314-4f17-a0d1-0ff2391d7f79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a67727c9142485fa2d09a36200f90ed, user_id=12041f02fb4146a2933846642764e8e9, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:42.582107 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e51ee0f-6314-4f17-a0d1-0ff2391d7f79 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:42.598179 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1046/2113] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:25:42 2022] PUT /identity/v3/projects/7a67727c9142485fa2d09a36200f90ed/users/12041f02fb4146a2933846642764e8e9/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:42.602201 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fa8870-936e-42dc-b131-84740dc810bf admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:42.612673 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fa8870-936e-42dc-b131-84740dc810bf admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:42.674562 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b737e84-5600-4e11-85a1-7adf0838a775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:42.675608 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b737e84-5600-4e11-85a1-7adf0838a775 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:42.676021 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b737e84-5600-4e11-85a1-7adf0838a775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:42.676428 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b737e84-5600-4e11-85a1-7adf0838a775 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:42.677653 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b737e84-5600-4e11-85a1-7adf0838a775 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:42.680195 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b737e84-5600-4e11-85a1-7adf0838a775 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:42.686031 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e51ee0f-6314-4f17-a0d1-0ff2391d7f79 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:42.688600 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-2e51ee0f-6314-4f17-a0d1-0ff2391d7f79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:42.697254 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1068/2114] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:25:42 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:42.703008 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b737e84-5600-4e11-85a1-7adf0838a775 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:42.707724 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e51ee0f-6314-4f17-a0d1-0ff2391d7f79 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:42.715461 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0228dd-bda6-428f-ac5c-f71d2927b73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:42.716830 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0228dd-bda6-428f-ac5c-f71d2927b73c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:42.717344 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0228dd-bda6-428f-ac5c-f71d2927b73c admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:42.717828 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0228dd-bda6-428f-ac5c-f71d2927b73c admin admin] PATH_INFO: `/v3/users/833a3035dcfb46d3b1efde8482a3b743` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:42.718833 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b737e84-5600-4e11-85a1-7adf0838a775 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:42.719612 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0228dd-bda6-428f-ac5c-f71d2927b73c admin admin] RBAC: Authorizing `identity:delete_user(user_id=833a3035dcfb46d3b1efde8482a3b743)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:42.721596 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0228dd-bda6-428f-ac5c-f71d2927b73c admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:42.723920 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682320f4-14b6-42eb-8fda-ff46c807fd79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:42.724973 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682320f4-14b6-42eb-8fda-ff46c807fd79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:42.725294 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682320f4-14b6-42eb-8fda-ff46c807fd79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:42.725566 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682320f4-14b6-42eb-8fda-ff46c807fd79 admin admin] PATH_INFO: `/v3/projects/7a67727c9142485fa2d09a36200f90ed/users/12041f02fb4146a2933846642764e8e9/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:42.727627 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682320f4-14b6-42eb-8fda-ff46c807fd79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a67727c9142485fa2d09a36200f90ed, user_id=12041f02fb4146a2933846642764e8e9, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:42.730374 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682320f4-14b6-42eb-8fda-ff46c807fd79 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:42.742707 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1069/2115] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:25:42 2022] PUT /identity/v3/projects/7a67727c9142485fa2d09a36200f90ed/users/12041f02fb4146a2933846642764e8e9/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:42.754851 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f48f51b-ef4c-4d13-bc3a-a4d31ca160f2 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:42.755254 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f48f51b-ef4c-4d13-bc3a-a4d31ca160f2 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:42.755441 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f48f51b-ef4c-4d13-bc3a-a4d31ca160f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:42.775492 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-2d0228dd-bda6-428f-ac5c-f71d2927b73c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '833a3035dcfb46d3b1efde8482a3b743'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:42.784610 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-5f48f51b-ef4c-4d13-bc3a-a4d31ca160f2 None None] MFA Rules not processed for user `12041f02fb4146a2933846642764e8e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:42.785855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-5f48f51b-ef4c-4d13-bc3a-a4d31ca160f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:42.788693 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-2d0228dd-bda6-428f-ac5c-f71d2927b73c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '833a3035dcfb46d3b1efde8482a3b743'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:42.798007 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-2d0228dd-bda6-428f-ac5c-f71d2927b73c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '833a3035dcfb46d3b1efde8482a3b743'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:42.806485 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-2d0228dd-bda6-428f-ac5c-f71d2927b73c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '833a3035dcfb46d3b1efde8482a3b743'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:42.809568 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-2d0228dd-bda6-428f-ac5c-f71d2927b73c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '833a3035dcfb46d3b1efde8482a3b743'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:42.816772 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1047/2116] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:25:42 2022] DELETE /identity/v3/users/833a3035dcfb46d3b1efde8482a3b743 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:42.821904 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d1074-7dca-4cde-a866-af2a8b42039d None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:42.822175 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d1074-7dca-4cde-a866-af2a8b42039d None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:42.822557 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d1074-7dca-4cde-a866-af2a8b42039d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:42.842790 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-b53d1074-7dca-4cde-a866-af2a8b42039d None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:42.843226 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:42.844207 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-b53d1074-7dca-4cde-a866-af2a8b42039d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:42.865986 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1070/2117] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:25:42 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:42.878411 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f48f51b-ef4c-4d13-bc3a-a4d31ca160f2 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:42.879812 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-5f48f51b-ef4c-4d13-bc3a-a4d31ca160f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:42.909984 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:42.915285 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1048/2118] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:25:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:42.925553 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53d1074-7dca-4cde-a866-af2a8b42039d None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:42.926994 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-b53d1074-7dca-4cde-a866-af2a8b42039d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:42.944304 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f48f51b-ef4c-4d13-bc3a-a4d31ca160f2 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:42.995770 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc89c475-89ef-4253-a3c0-da6b44ca0fcd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:42.997772 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53d1074-7dca-4cde-a866-af2a8b42039d None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:42.997902 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc89c475-89ef-4253-a3c0-da6b44ca0fcd service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:42.997902 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc89c475-89ef-4253-a3c0-da6b44ca0fcd service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:42.997902 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc89c475-89ef-4253-a3c0-da6b44ca0fcd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:43.001345 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74743ef5-655a-4e84-9681-c2836c04bfe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:43.002092 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74743ef5-655a-4e84-9681-c2836c04bfe3 service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:43.002565 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74743ef5-655a-4e84-9681-c2836c04bfe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:43.002916 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74743ef5-655a-4e84-9681-c2836c04bfe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:43.015244 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74743ef5-655a-4e84-9681-c2836c04bfe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:43.016915 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74743ef5-655a-4e84-9681-c2836c04bfe3 service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:43.038364 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1049/2119] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:25:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:25:43.070436 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc89c475-89ef-4253-a3c0-da6b44ca0fcd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:43.071890 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc89c475-89ef-4253-a3c0-da6b44ca0fcd service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:43.081721 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1071/2120] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:25:42 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:25:43.177330 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74743ef5-655a-4e84-9681-c2836c04bfe3 service neutron] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:43.186838 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74743ef5-655a-4e84-9681-c2836c04bfe3 service neutron] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:43.190653 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc102714-ecae-4cfc-8e6b-8ea3c1758e98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:43.191313 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc102714-ecae-4cfc-8e6b-8ea3c1758e98 service cinder] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:43.191580 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc102714-ecae-4cfc-8e6b-8ea3c1758e98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:43.191850 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc102714-ecae-4cfc-8e6b-8ea3c1758e98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:43.193915 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-bc102714-ecae-4cfc-8e6b-8ea3c1758e98 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:43.286923 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc102714-ecae-4cfc-8e6b-8ea3c1758e98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:43.288554 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc102714-ecae-4cfc-8e6b-8ea3c1758e98 service cinder] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:43.304272 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1050/2121] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:25:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:25:43.505437 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc89c475-89ef-4253-a3c0-da6b44ca0fcd service cinder] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:43.506930 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-bc89c475-89ef-4253-a3c0-da6b44ca0fcd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:43.521859 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc89c475-89ef-4253-a3c0-da6b44ca0fcd service cinder] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:43.527706 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac0c447-acff-46fd-9e9a-ca09d717bdc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:43.528622 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0c447-acff-46fd-9e9a-ca09d717bdc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:43.529166 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0c447-acff-46fd-9e9a-ca09d717bdc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:43.529359 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0c447-acff-46fd-9e9a-ca09d717bdc4 admin admin] PATH_INFO: `/v3/projects/1d499026e5ce4fd1b94c2063e4535957` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:43.532837 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac0c447-acff-46fd-9e9a-ca09d717bdc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d499026e5ce4fd1b94c2063e4535957)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:43.541509 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac0c447-acff-46fd-9e9a-ca09d717bdc4 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:43.611822 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc102714-ecae-4cfc-8e6b-8ea3c1758e98 service cinder] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:43.613458 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3ac0c447-acff-46fd-9e9a-ca09d717bdc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d499026e5ce4fd1b94c2063e4535957', 'request_id': 'req-3ac0c447-acff-46fd-9e9a-ca09d717bdc4'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:43.613609 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-bc102714-ecae-4cfc-8e6b-8ea3c1758e98 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:43.615042 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:43.622862 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3ac0c447-acff-46fd-9e9a-ca09d717bdc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d499026e5ce4fd1b94c2063e4535957', 'request_id': 'req-3ac0c447-acff-46fd-9e9a-ca09d717bdc4'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:43.628539 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc102714-ecae-4cfc-8e6b-8ea3c1758e98 service cinder] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:43.633932 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3ac0c447-acff-46fd-9e9a-ca09d717bdc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d499026e5ce4fd1b94c2063e4535957', 'request_id': 'req-3ac0c447-acff-46fd-9e9a-ca09d717bdc4'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:43.646740 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3ac0c447-acff-46fd-9e9a-ca09d717bdc4 admin admin] The token cache is being invalidate because project 1d499026e5ce4fd1b94c2063e4535957 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:25:43.647025 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3ac0c447-acff-46fd-9e9a-ca09d717bdc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:43.647659 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3ac0c447-acff-46fd-9e9a-ca09d717bdc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:43.649522 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1072/2122] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:25:43 2022] DELETE /identity/v3/projects/1d499026e5ce4fd1b94c2063e4535957 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:43.693786 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d9900-9123-429c-bf5a-0087806d7c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:43.694711 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d9900-9123-429c-bf5a-0087806d7c52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:43.694994 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d9900-9123-429c-bf5a-0087806d7c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:43.695309 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d9900-9123-429c-bf5a-0087806d7c52 admin admin] PATH_INFO: `/v3/users/4fe482c1db9e4fabb0b0f6c9912d5c21` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:43.696607 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4d9900-9123-429c-bf5a-0087806d7c52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fe482c1db9e4fabb0b0f6c9912d5c21)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:43.698648 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4d9900-9123-429c-bf5a-0087806d7c52 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:43.751633 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:43.754724 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-af4d9900-9123-429c-bf5a-0087806d7c52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fe482c1db9e4fabb0b0f6c9912d5c21'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:43.769950 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-af4d9900-9123-429c-bf5a-0087806d7c52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fe482c1db9e4fabb0b0f6c9912d5c21'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:43.778477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-af4d9900-9123-429c-bf5a-0087806d7c52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fe482c1db9e4fabb0b0f6c9912d5c21'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:43.786825 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-af4d9900-9123-429c-bf5a-0087806d7c52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fe482c1db9e4fabb0b0f6c9912d5c21'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:43.787560 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-af4d9900-9123-429c-bf5a-0087806d7c52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fe482c1db9e4fabb0b0f6c9912d5c21'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:43.793899 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1051/2123] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:25:43 2022] DELETE /identity/v3/users/4fe482c1db9e4fabb0b0f6c9912d5c21 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:43.798800 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef1cdc-a1e7-4f85-b1d9-a9e414d3c681 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:43.799086 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef1cdc-a1e7-4f85-b1d9-a9e414d3c681 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:43.799366 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef1cdc-a1e7-4f85-b1d9-a9e414d3c681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:43.819094 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-59ef1cdc-a1e7-4f85-b1d9-a9e414d3c681 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:43.820325 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-59ef1cdc-a1e7-4f85-b1d9-a9e414d3c681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:43.891770 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1073/2124] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:25:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:43.902279 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d9900-9123-429c-bf5a-0087806d7c52 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:43.904764 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-af4d9900-9123-429c-bf5a-0087806d7c52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:43.966229 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d9900-9123-429c-bf5a-0087806d7c52 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:43.971134 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a973f9d3-12dd-4c34-b017-d8c13c0d8c37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:43.971875 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973f9d3-12dd-4c34-b017-d8c13c0d8c37 service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:43.972243 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973f9d3-12dd-4c34-b017-d8c13c0d8c37 service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:43.972580 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973f9d3-12dd-4c34-b017-d8c13c0d8c37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:43.982736 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a973f9d3-12dd-4c34-b017-d8c13c0d8c37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:43.984406 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a973f9d3-12dd-4c34-b017-d8c13c0d8c37 service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:43.995020 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1052/2125] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:25:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:25:44.276019 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ef1cdc-a1e7-4f85-b1d9-a9e414d3c681 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:44.277573 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-59ef1cdc-a1e7-4f85-b1d9-a9e414d3c681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:44.291072 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ef1cdc-a1e7-4f85-b1d9-a9e414d3c681 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:44.296451 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4174d5-243a-4347-8846-cc5427499afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:44.297465 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4174d5-243a-4347-8846-cc5427499afc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:44.297926 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4174d5-243a-4347-8846-cc5427499afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:44.298354 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4174d5-243a-4347-8846-cc5427499afc admin admin] PATH_INFO: `/v3/projects/f2e524619d7f49478237e5bb41115c11` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:44.299866 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4174d5-243a-4347-8846-cc5427499afc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2e524619d7f49478237e5bb41115c11)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:44.302670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4174d5-243a-4347-8846-cc5427499afc admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:44.353948 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-7d4174d5-243a-4347-8846-cc5427499afc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2e524619d7f49478237e5bb41115c11', 'request_id': 'req-7d4174d5-243a-4347-8846-cc5427499afc'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:44.361476 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-7d4174d5-243a-4347-8846-cc5427499afc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2e524619d7f49478237e5bb41115c11', 'request_id': 'req-7d4174d5-243a-4347-8846-cc5427499afc'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:44.370055 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-7d4174d5-243a-4347-8846-cc5427499afc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2e524619d7f49478237e5bb41115c11', 'request_id': 'req-7d4174d5-243a-4347-8846-cc5427499afc'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:44.381090 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-7d4174d5-243a-4347-8846-cc5427499afc admin admin] The token cache is being invalidate because project f2e524619d7f49478237e5bb41115c11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:25:44.381378 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-7d4174d5-243a-4347-8846-cc5427499afc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:44.382544 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-7d4174d5-243a-4347-8846-cc5427499afc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:44.384335 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1074/2126] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:25:44 2022] DELETE /identity/v3/projects/f2e524619d7f49478237e5bb41115c11 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:44.388563 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a973f9d3-12dd-4c34-b017-d8c13c0d8c37 service neutron] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:44.398443 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a973f9d3-12dd-4c34-b017-d8c13c0d8c37 service neutron] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:44.490655 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5088cfd-56c0-40b6-999f-c35d640903d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:44.491554 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5088cfd-56c0-40b6-999f-c35d640903d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:44.491836 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5088cfd-56c0-40b6-999f-c35d640903d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:44.492160 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5088cfd-56c0-40b6-999f-c35d640903d5 admin admin] PATH_INFO: `/v3/users/12041f02fb4146a2933846642764e8e9` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:44.493774 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5088cfd-56c0-40b6-999f-c35d640903d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12041f02fb4146a2933846642764e8e9)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:44.495861 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5088cfd-56c0-40b6-999f-c35d640903d5 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:44.595293 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-c5088cfd-56c0-40b6-999f-c35d640903d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12041f02fb4146a2933846642764e8e9'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:44.627747 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-c5088cfd-56c0-40b6-999f-c35d640903d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12041f02fb4146a2933846642764e8e9'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:44.630932 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:44.650501 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-c5088cfd-56c0-40b6-999f-c35d640903d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12041f02fb4146a2933846642764e8e9'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:44.658683 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-c5088cfd-56c0-40b6-999f-c35d640903d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12041f02fb4146a2933846642764e8e9'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:44.659391 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-c5088cfd-56c0-40b6-999f-c35d640903d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12041f02fb4146a2933846642764e8e9'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:44.667738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1053/2127] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:25:44 2022] DELETE /identity/v3/users/12041f02fb4146a2933846642764e8e9 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:45.396600 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4174d5-243a-4347-8846-cc5427499afc admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:45.398206 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-7d4174d5-243a-4347-8846-cc5427499afc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:45.413190 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4174d5-243a-4347-8846-cc5427499afc admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:45.529669 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75cf875-4cc6-4499-8c55-8120f7597621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:45.530820 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75cf875-4cc6-4499-8c55-8120f7597621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:45.531289 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75cf875-4cc6-4499-8c55-8120f7597621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:45.531573 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75cf875-4cc6-4499-8c55-8120f7597621 admin admin] PATH_INFO: `/v3/projects/7a67727c9142485fa2d09a36200f90ed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:45.533496 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75cf875-4cc6-4499-8c55-8120f7597621 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a67727c9142485fa2d09a36200f90ed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:45.535448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75cf875-4cc6-4499-8c55-8120f7597621 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:45.539863 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:45.600160 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-d75cf875-4cc6-4499-8c55-8120f7597621 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a67727c9142485fa2d09a36200f90ed', 'request_id': 'req-d75cf875-4cc6-4499-8c55-8120f7597621'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:45.612022 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-d75cf875-4cc6-4499-8c55-8120f7597621 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a67727c9142485fa2d09a36200f90ed', 'request_id': 'req-d75cf875-4cc6-4499-8c55-8120f7597621'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:45.624597 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-d75cf875-4cc6-4499-8c55-8120f7597621 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a67727c9142485fa2d09a36200f90ed', 'request_id': 'req-d75cf875-4cc6-4499-8c55-8120f7597621'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:45.641988 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-d75cf875-4cc6-4499-8c55-8120f7597621 admin admin] The token cache is being invalidate because project 7a67727c9142485fa2d09a36200f90ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:25:45.643170 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-d75cf875-4cc6-4499-8c55-8120f7597621 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:45.643902 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-d75cf875-4cc6-4499-8c55-8120f7597621 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:45.646216 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1075/2128] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:25:45 2022] DELETE /identity/v3/projects/7a67727c9142485fa2d09a36200f90ed => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:45.651861 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0612a037-1757-4ad4-ad23-48245c1c6dd5 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:45.652372 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0612a037-1757-4ad4-ad23-48245c1c6dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:45.652473 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0612a037-1757-4ad4-ad23-48245c1c6dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:45.672646 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-0612a037-1757-4ad4-ad23-48245c1c6dd5 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:45.674158 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-0612a037-1757-4ad4-ad23-48245c1c6dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:45.809389 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1054/2129] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:25:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:25:45.818578 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5546ec65-356d-48e0-af1a-7b247e4ea3a4 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:45.819070 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5546ec65-356d-48e0-af1a-7b247e4ea3a4 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:45.819358 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5546ec65-356d-48e0-af1a-7b247e4ea3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:45.841672 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-5546ec65-356d-48e0-af1a-7b247e4ea3a4 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:45.843024 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-5546ec65-356d-48e0-af1a-7b247e4ea3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:45.872773 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1076/2130] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:25:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:45.874237 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0612a037-1757-4ad4-ad23-48245c1c6dd5 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:45.891051 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0612a037-1757-4ad4-ad23-48245c1c6dd5 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:45.899431 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da204167-af11-4f72-8d3d-fd484ff93d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:45.900162 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da204167-af11-4f72-8d3d-fd484ff93d7f admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:45.900836 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da204167-af11-4f72-8d3d-fd484ff93d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:45.901284 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da204167-af11-4f72-8d3d-fd484ff93d7f admin admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:45.902533 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da204167-af11-4f72-8d3d-fd484ff93d7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:25:45.902845 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da204167-af11-4f72-8d3d-fd484ff93d7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:45.904680 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da204167-af11-4f72-8d3d-fd484ff93d7f admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:45.921754 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1055/2131] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:25:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:25:45.924610 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5546ec65-356d-48e0-af1a-7b247e4ea3a4 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:45.939345 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5546ec65-356d-48e0-af1a-7b247e4ea3a4 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:45.945509 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb627db-1da4-4257-a248-4730d13f6f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:45.946546 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb627db-1da4-4257-a248-4730d13f6f1f admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:45.946826 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb627db-1da4-4257-a248-4730d13f6f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:45.947115 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb627db-1da4-4257-a248-4730d13f6f1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:45.948137 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb627db-1da4-4257-a248-4730d13f6f1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:45.950907 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb627db-1da4-4257-a248-4730d13f6f1f admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:45.998662 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1077/2132] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:25:45 2022] POST /identity/v3/projects => generated 385 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:25:46.002873 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da204167-af11-4f72-8d3d-fd484ff93d7f admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:46.019939 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da204167-af11-4f72-8d3d-fd484ff93d7f admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:46.117142 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb232e00-4f49-4863-8be9-8c1fcd70d895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:46.118033 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb232e00-4f49-4863-8be9-8c1fcd70d895 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:46.119685 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb232e00-4f49-4863-8be9-8c1fcd70d895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:46.119685 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb232e00-4f49-4863-8be9-8c1fcd70d895 admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:46.120100 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb232e00-4f49-4863-8be9-8c1fcd70d895 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:46.122531 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb232e00-4f49-4863-8be9-8c1fcd70d895 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:46.153995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb627db-1da4-4257-a248-4730d13f6f1f admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:46.155184 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1056/2133] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:25:46 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:25:46.168439 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb627db-1da4-4257-a248-4730d13f6f1f admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:46.186087 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f82aa63-8081-4317-b6c9-ed4c209ab3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:46.186784 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82aa63-8081-4317-b6c9-ed4c209ab3e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:46.187110 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82aa63-8081-4317-b6c9-ed4c209ab3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:46.187678 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82aa63-8081-4317-b6c9-ed4c209ab3e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:46.188162 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f82aa63-8081-4317-b6c9-ed4c209ab3e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:46.195027 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f82aa63-8081-4317-b6c9-ed4c209ab3e5 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:46.221397 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1078/2134] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:25:46 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:46.225694 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb232e00-4f49-4863-8be9-8c1fcd70d895 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:46.240591 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb232e00-4f49-4863-8be9-8c1fcd70d895 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:46.245826 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf432d0f-ebc5-4a81-8c24-5e6a8fe4faf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:46.246964 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf432d0f-ebc5-4a81-8c24-5e6a8fe4faf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:46.247322 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf432d0f-ebc5-4a81-8c24-5e6a8fe4faf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:46.247833 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf432d0f-ebc5-4a81-8c24-5e6a8fe4faf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:46.248666 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf432d0f-ebc5-4a81-8c24-5e6a8fe4faf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:46.250939 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf432d0f-ebc5-4a81-8c24-5e6a8fe4faf5 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:46.278047 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1057/2135] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:25:46 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:46.282746 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f82aa63-8081-4317-b6c9-ed4c209ab3e5 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:46.307908 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f82aa63-8081-4317-b6c9-ed4c209ab3e5 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:46.314493 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f2132-2ad5-4e7a-8e1f-961a8ded58c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:46.315465 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f2132-2ad5-4e7a-8e1f-961a8ded58c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:46.315880 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f2132-2ad5-4e7a-8e1f-961a8ded58c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:46.316163 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f2132-2ad5-4e7a-8e1f-961a8ded58c8 admin admin] PATH_INFO: `/v3/projects/3c974d375f4544b6b66af647cbe117ec/users/9e3a6109c25c4f61b4a4eef8888f3cfc/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:46.339217 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f2132-2ad5-4e7a-8e1f-961a8ded58c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c974d375f4544b6b66af647cbe117ec, user_id=9e3a6109c25c4f61b4a4eef8888f3cfc, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:46.342668 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f2132-2ad5-4e7a-8e1f-961a8ded58c8 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:46.361987 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1079/2136] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:25:46 2022] PUT /identity/v3/projects/3c974d375f4544b6b66af647cbe117ec/users/9e3a6109c25c4f61b4a4eef8888f3cfc/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:46.372658 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6836e-64b1-44cb-8754-cccdfcdfa008 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:46.372944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6836e-64b1-44cb-8754-cccdfcdfa008 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:46.373241 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6836e-64b1-44cb-8754-cccdfcdfa008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:46.413283 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-d4d6836e-64b1-44cb-8754-cccdfcdfa008 None None] MFA Rules not processed for user `9e3a6109c25c4f61b4a4eef8888f3cfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:46.414517 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-d4d6836e-64b1-44cb-8754-cccdfcdfa008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:46.489771 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:46.541464 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1058/2137] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:25:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:46.556427 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f2132-2ad5-4e7a-8e1f-961a8ded58c8 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:46.561268 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-d32f2132-2ad5-4e7a-8e1f-961a8ded58c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:46.681543 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f2132-2ad5-4e7a-8e1f-961a8ded58c8 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:46.686526 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50322fd1-2052-4725-a3dc-3102cc22b0dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:46.687193 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50322fd1-2052-4725-a3dc-3102cc22b0dd service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:46.687476 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50322fd1-2052-4725-a3dc-3102cc22b0dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:46.687754 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50322fd1-2052-4725-a3dc-3102cc22b0dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:46.697737 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50322fd1-2052-4725-a3dc-3102cc22b0dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:46.699278 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50322fd1-2052-4725-a3dc-3102cc22b0dd service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:46.711059 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1080/2138] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:25:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:25:48.876068 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f163ec-26de-4252-8455-5cfd9ebd9a06 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:48.876588 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f163ec-26de-4252-8455-5cfd9ebd9a06 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:48.876588 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f163ec-26de-4252-8455-5cfd9ebd9a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:48.901302 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-03f163ec-26de-4252-8455-5cfd9ebd9a06 None None] MFA Rules not processed for user `a094273e734342eca2bdf159fab8a8d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:48.901986 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-03f163ec-26de-4252-8455-5cfd9ebd9a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:48.971396 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1059/2139] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:25:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:52.702402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eeb10f-b642-497b-b24a-27bc7514cdd3 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:52.704474 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eeb10f-b642-497b-b24a-27bc7514cdd3 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:52.705006 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eeb10f-b642-497b-b24a-27bc7514cdd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:52.744990 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-11eeb10f-b642-497b-b24a-27bc7514cdd3 None None] MFA Rules not processed for user `86a18e313e4944748aef58d54bfe7b13`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:52.746279 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-11eeb10f-b642-497b-b24a-27bc7514cdd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:52.772286 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:52.856821 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1081/2140] 10.222.0.50 () {58 vars in 1009 bytes} [Fri May 27 03:25:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:52.936618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f163ec-26de-4252-8455-5cfd9ebd9a06 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:52.938974 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-03f163ec-26de-4252-8455-5cfd9ebd9a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:53.044691 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f163ec-26de-4252-8455-5cfd9ebd9a06 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:53.071191 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc220f-90ac-4701-99c2-fc0ca8a8dd09 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de53e2f0fba4e0fac70887067c1963d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:53.071870 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc220f-90ac-4701-99c2-fc0ca8a8dd09 service swift] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:53.072147 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc220f-90ac-4701-99c2-fc0ca8a8dd09 service swift] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:53.072767 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc220f-90ac-4701-99c2-fc0ca8a8dd09 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:53.085248 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc220f-90ac-4701-99c2-fc0ca8a8dd09 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:53.086907 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc220f-90ac-4701-99c2-fc0ca8a8dd09 service swift] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:53.100160 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1060/2141] 10.222.0.50 () {60 vars in 1384 bytes} [Fri May 27 03:25:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:25:56.063749 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eeb10f-b642-497b-b24a-27bc7514cdd3 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:56.292354 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eeb10f-b642-497b-b24a-27bc7514cdd3 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:56.304709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ad60d-b453-4c16-9ff8-227c580f5101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:56.305656 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ad60d-b453-4c16-9ff8-227c580f5101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:56.305942 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ad60d-b453-4c16-9ff8-227c580f5101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:56.306219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ad60d-b453-4c16-9ff8-227c580f5101 admin admin] PATH_INFO: `/v3/users/9e3a6109c25c4f61b4a4eef8888f3cfc` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:56.307569 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450ad60d-b453-4c16-9ff8-227c580f5101 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e3a6109c25c4f61b4a4eef8888f3cfc)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:56.310212 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450ad60d-b453-4c16-9ff8-227c580f5101 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:56.448462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-450ad60d-b453-4c16-9ff8-227c580f5101 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e3a6109c25c4f61b4a4eef8888f3cfc'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:56.449214 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-450ad60d-b453-4c16-9ff8-227c580f5101 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e3a6109c25c4f61b4a4eef8888f3cfc'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:56.473826 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-450ad60d-b453-4c16-9ff8-227c580f5101 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e3a6109c25c4f61b4a4eef8888f3cfc'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:56.475024 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-450ad60d-b453-4c16-9ff8-227c580f5101 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e3a6109c25c4f61b4a4eef8888f3cfc'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:56.500032 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-450ad60d-b453-4c16-9ff8-227c580f5101 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e3a6109c25c4f61b4a4eef8888f3cfc'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:56.531821 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1082/2142] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:25:56 2022] DELETE /identity/v3/users/9e3a6109c25c4f61b4a4eef8888f3cfc => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:56.537026 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6090be0-f7c2-4f14-9cef-8ca8954497b1 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:56.537026 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6090be0-f7c2-4f14-9cef-8ca8954497b1 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:56.537026 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6090be0-f7c2-4f14-9cef-8ca8954497b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:56.568903 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-f6090be0-f7c2-4f14-9cef-8ca8954497b1 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:56.570319 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-f6090be0-f7c2-4f14-9cef-8ca8954497b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:56.706268 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1061/2143] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:25:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:56.723332 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ad60d-b453-4c16-9ff8-227c580f5101 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:56.725063 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-450ad60d-b453-4c16-9ff8-227c580f5101 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:56.822229 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:56.845649 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ad60d-b453-4c16-9ff8-227c580f5101 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:56.851343 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38efd411-1208-466d-be9f-249aea9a7ffd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:56.852322 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38efd411-1208-466d-be9f-249aea9a7ffd service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:56.852764 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38efd411-1208-466d-be9f-249aea9a7ffd service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:56.853323 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38efd411-1208-466d-be9f-249aea9a7ffd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:56.864729 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38efd411-1208-466d-be9f-249aea9a7ffd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:56.867085 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38efd411-1208-466d-be9f-249aea9a7ffd service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:56.878913 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1083/2144] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:25:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:25:56.970546 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6090be0-f7c2-4f14-9cef-8ca8954497b1 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:56.976260 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-f6090be0-f7c2-4f14-9cef-8ca8954497b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:57.003086 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6090be0-f7c2-4f14-9cef-8ca8954497b1 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:57.010218 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1715516-deaa-48bd-8994-75cef6b3f817 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:57.013425 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1715516-deaa-48bd-8994-75cef6b3f817 service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:57.013716 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1715516-deaa-48bd-8994-75cef6b3f817 service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:57.014236 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1715516-deaa-48bd-8994-75cef6b3f817 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:57.019995 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-d1715516-deaa-48bd-8994-75cef6b3f817 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:57.047783 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:57.188175 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1715516-deaa-48bd-8994-75cef6b3f817 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:57.189722 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1715516-deaa-48bd-8994-75cef6b3f817 service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:57.219137 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1062/2145] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:25:56 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:25:57.838380 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38efd411-1208-466d-be9f-249aea9a7ffd service neutron] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:57.840772 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-38efd411-1208-466d-be9f-249aea9a7ffd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:57.874585 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38efd411-1208-466d-be9f-249aea9a7ffd service neutron] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:57.900652 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50d23a2-9ecd-4081-86cb-c8a64a7e473b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:57.901632 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d23a2-9ecd-4081-86cb-c8a64a7e473b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:57.901914 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d23a2-9ecd-4081-86cb-c8a64a7e473b admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:57.902184 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d23a2-9ecd-4081-86cb-c8a64a7e473b admin admin] PATH_INFO: `/v3/projects/3c974d375f4544b6b66af647cbe117ec` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:57.903389 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50d23a2-9ecd-4081-86cb-c8a64a7e473b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c974d375f4544b6b66af647cbe117ec)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:57.905599 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50d23a2-9ecd-4081-86cb-c8a64a7e473b admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:58.000531 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-b50d23a2-9ecd-4081-86cb-c8a64a7e473b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c974d375f4544b6b66af647cbe117ec', 'request_id': 'req-b50d23a2-9ecd-4081-86cb-c8a64a7e473b'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:58.011043 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-b50d23a2-9ecd-4081-86cb-c8a64a7e473b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c974d375f4544b6b66af647cbe117ec', 'request_id': 'req-b50d23a2-9ecd-4081-86cb-c8a64a7e473b'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:58.028926 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-b50d23a2-9ecd-4081-86cb-c8a64a7e473b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c974d375f4544b6b66af647cbe117ec', 'request_id': 'req-b50d23a2-9ecd-4081-86cb-c8a64a7e473b'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:58.054872 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-b50d23a2-9ecd-4081-86cb-c8a64a7e473b admin admin] The token cache is being invalidate because project 3c974d375f4544b6b66af647cbe117ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:25:58.055309 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-b50d23a2-9ecd-4081-86cb-c8a64a7e473b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:58.057359 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-b50d23a2-9ecd-4081-86cb-c8a64a7e473b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:25:58.065329 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1084/2146] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:25:57 2022] DELETE /identity/v3/projects/3c974d375f4544b6b66af647cbe117ec => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:58.066127 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef50d5bb-0d90-4848-81bf-7cd745815c2f None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:58.067582 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef50d5bb-0d90-4848-81bf-7cd745815c2f None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:58.068662 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef50d5bb-0d90-4848-81bf-7cd745815c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:58.095002 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-ef50d5bb-0d90-4848-81bf-7cd745815c2f None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:58.096511 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-ef50d5bb-0d90-4848-81bf-7cd745815c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:58.192554 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1063/2147] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:25:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:58.201286 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfcd6aa-b42e-4ae1-b646-11c7a80033af None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:58.201648 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfcd6aa-b42e-4ae1-b646-11c7a80033af None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:58.202746 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfcd6aa-b42e-4ae1-b646-11c7a80033af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:58.240054 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-fdfcd6aa-b42e-4ae1-b646-11c7a80033af None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:58.241512 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-fdfcd6aa-b42e-4ae1-b646-11c7a80033af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:58.264219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1085/2148] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:25:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:58.269154 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef50d5bb-0d90-4848-81bf-7cd745815c2f None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:58.285546 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef50d5bb-0d90-4848-81bf-7cd745815c2f None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:58.292219 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0c5da-e793-49b8-b493-18768b89c424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:58.300688 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0c5da-e793-49b8-b493-18768b89c424 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:58.301018 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0c5da-e793-49b8-b493-18768b89c424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:58.301264 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade0c5da-e793-49b8-b493-18768b89c424 admin admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:58.302860 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade0c5da-e793-49b8-b493-18768b89c424 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:25:58.303334 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade0c5da-e793-49b8-b493-18768b89c424 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:58.305457 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade0c5da-e793-49b8-b493-18768b89c424 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:58.340657 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1064/2149] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:25:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:25:58.345566 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfcd6aa-b42e-4ae1-b646-11c7a80033af None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:58.364276 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfcd6aa-b42e-4ae1-b646-11c7a80033af None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:58.378455 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce4e2a5-d44d-4fdc-8c19-e3c9e39e7c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:58.379253 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce4e2a5-d44d-4fdc-8c19-e3c9e39e7c21 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:58.380308 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce4e2a5-d44d-4fdc-8c19-e3c9e39e7c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:58.380651 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce4e2a5-d44d-4fdc-8c19-e3c9e39e7c21 admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:58.381457 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce4e2a5-d44d-4fdc-8c19-e3c9e39e7c21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:58.383470 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce4e2a5-d44d-4fdc-8c19-e3c9e39e7c21 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:58.456558 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1086/2150] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:25:58 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:25:58.457637 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0c5da-e793-49b8-b493-18768b89c424 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:58.471463 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade0c5da-e793-49b8-b493-18768b89c424 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:58.591575 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b16b2-5d11-4fe6-97a0-d7c726298df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:58.592359 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b16b2-5d11-4fe6-97a0-d7c726298df3 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:58.592630 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b16b2-5d11-4fe6-97a0-d7c726298df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:58.592946 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b16b2-5d11-4fe6-97a0-d7c726298df3 admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:58.593773 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96b16b2-5d11-4fe6-97a0-d7c726298df3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:58.595722 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96b16b2-5d11-4fe6-97a0-d7c726298df3 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:58.638355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1065/2151] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:25:58 2022] POST /identity/v3/users => generated 289 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:25:58.639644 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce4e2a5-d44d-4fdc-8c19-e3c9e39e7c21 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:58.660271 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce4e2a5-d44d-4fdc-8c19-e3c9e39e7c21 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:58.667804 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357ee90-40b9-4e81-90ed-667ce5b30b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:58.668765 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357ee90-40b9-4e81-90ed-667ce5b30b87 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:58.669797 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357ee90-40b9-4e81-90ed-667ce5b30b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:58.670342 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357ee90-40b9-4e81-90ed-667ce5b30b87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:58.671464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2357ee90-40b9-4e81-90ed-667ce5b30b87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:58.673342 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2357ee90-40b9-4e81-90ed-667ce5b30b87 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:58.691069 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1087/2152] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:58.696367 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b16b2-5d11-4fe6-97a0-d7c726298df3 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:58.714389 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b16b2-5d11-4fe6-97a0-d7c726298df3 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:58.727909 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec9aaa-728c-44d9-85fa-7991fcfe640f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:58.729697 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec9aaa-728c-44d9-85fa-7991fcfe640f admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:58.730462 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec9aaa-728c-44d9-85fa-7991fcfe640f admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:58.730965 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec9aaa-728c-44d9-85fa-7991fcfe640f admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:58.731920 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fec9aaa-728c-44d9-85fa-7991fcfe640f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:58.733802 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fec9aaa-728c-44d9-85fa-7991fcfe640f admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:58.746142 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1066/2153] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:25:58.752134 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357ee90-40b9-4e81-90ed-667ce5b30b87 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:58.787256 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357ee90-40b9-4e81-90ed-667ce5b30b87 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:58.808387 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34714e8-1a3a-40ac-9a70-afcb760292f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:58.809964 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34714e8-1a3a-40ac-9a70-afcb760292f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:58.810506 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34714e8-1a3a-40ac-9a70-afcb760292f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:58.811018 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34714e8-1a3a-40ac-9a70-afcb760292f3 admin admin] PATH_INFO: `/v3/projects/76ca9458e9fb4201b02963e6d9be9db7/users/feb7b7347fab4e6ea613c49c9be59eaf/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:58.842575 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34714e8-1a3a-40ac-9a70-afcb760292f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76ca9458e9fb4201b02963e6d9be9db7, user_id=feb7b7347fab4e6ea613c49c9be59eaf, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:58.848227 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34714e8-1a3a-40ac-9a70-afcb760292f3 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:58.930635 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1088/2154] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:25:58 2022] PUT /identity/v3/projects/76ca9458e9fb4201b02963e6d9be9db7/users/feb7b7347fab4e6ea613c49c9be59eaf/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:25:58.932257 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab2ae0-6a9d-4aa1-85ec-0b4b1c27d6e6 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:58.932551 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab2ae0-6a9d-4aa1-85ec-0b4b1c27d6e6 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:58.932832 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab2ae0-6a9d-4aa1-85ec-0b4b1c27d6e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:58.983090 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:25:58.997541 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-92ab2ae0-6a9d-4aa1-85ec-0b4b1c27d6e6 None None] MFA Rules not processed for user `feb7b7347fab4e6ea613c49c9be59eaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:25:58.998869 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-92ab2ae0-6a9d-4aa1-85ec-0b4b1c27d6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:59.169968 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1067/2155] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:25:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:25:59.178385 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34714e8-1a3a-40ac-9a70-afcb760292f3 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:25:59.180812 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-d34714e8-1a3a-40ac-9a70-afcb760292f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:25:59.391086 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34714e8-1a3a-40ac-9a70-afcb760292f3 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:25:59.401270 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7a514a-4580-4bbe-a838-a90e1ff152ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:25:59.401943 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a514a-4580-4bbe-a838-a90e1ff152ad service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:25:59.402227 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a514a-4580-4bbe-a838-a90e1ff152ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:25:59.402500 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a514a-4580-4bbe-a838-a90e1ff152ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:25:59.436915 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7a514a-4580-4bbe-a838-a90e1ff152ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:25:59.440064 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7a514a-4580-4bbe-a838-a90e1ff152ad service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:25:59.465186 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1089/2156] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:25:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:26:09.158574 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=26 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:09.449616 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:18.186116 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab2ae0-6a9d-4aa1-85ec-0b4b1c27d6e6 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:18.188196 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-92ab2ae0-6a9d-4aa1-85ec-0b4b1c27d6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:18.326128 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab2ae0-6a9d-4aa1-85ec-0b4b1c27d6e6 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:18.331045 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaaf47e-a479-4fda-899d-754931a5795e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30963f2f464e4f54a8aea0175370a235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:18.331798 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaaf47e-a479-4fda-899d-754931a5795e service placement] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:18.332004 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaaf47e-a479-4fda-899d-754931a5795e service placement] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:18.332235 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaaf47e-a479-4fda-899d-754931a5795e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:18.334294 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-2eaaf47e-a479-4fda-899d-754931a5795e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:18.347064 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaaf47e-a479-4fda-899d-754931a5795e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:18.348883 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaaf47e-a479-4fda-899d-754931a5795e service placement] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:18.369233 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1068/2157] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:26:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:26:19.611162 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683bba7-7272-49e0-b244-19c9f905fc48 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:19.611609 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683bba7-7272-49e0-b244-19c9f905fc48 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:19.611719 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a683bba7-7272-49e0-b244-19c9f905fc48 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:19.614125 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1090/2158] 10.222.0.50 () {54 vars in 921 bytes} [Fri May 27 03:26:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:26:19.620068 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb89c7a-1535-4f0c-9094-ae3a10026662 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:19.620912 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb89c7a-1535-4f0c-9094-ae3a10026662 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:19.621395 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb89c7a-1535-4f0c-9094-ae3a10026662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:19.684907 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-4fb89c7a-1535-4f0c-9094-ae3a10026662 None None] MFA Rules not processed for user `1a699988f08a4001b3b7d764f01f4fc5`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:19.686320 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-4fb89c7a-1535-4f0c-9094-ae3a10026662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:19.789398 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1069/2159] 10.222.0.50 () {58 vars in 1018 bytes} [Fri May 27 03:26:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:26:19.802531 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683bba7-7272-49e0-b244-19c9f905fc48 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:19.804439 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-a683bba7-7272-49e0-b244-19c9f905fc48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:19.926081 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a683bba7-7272-49e0-b244-19c9f905fc48 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:19.930408 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe33f8ab-0b9b-4ae7-90f8-09b219a44cd0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3c9c5c8d2e4376b1b70d974d4cb851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:19.932836 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33f8ab-0b9b-4ae7-90f8-09b219a44cd0 service glance] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:19.932836 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33f8ab-0b9b-4ae7-90f8-09b219a44cd0 service glance] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:19.932836 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33f8ab-0b9b-4ae7-90f8-09b219a44cd0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:19.946722 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe33f8ab-0b9b-4ae7-90f8-09b219a44cd0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:19.948533 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe33f8ab-0b9b-4ae7-90f8-09b219a44cd0 service glance] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:19.961477 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1091/2160] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:26:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:26:20.035606 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f82491-4dec-4d3a-b4c5-fc8e2eaf2cce None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:20.036155 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f82491-4dec-4d3a-b4c5-fc8e2eaf2cce None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:20.036437 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f82491-4dec-4d3a-b4c5-fc8e2eaf2cce None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:20.037852 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1070/2161] 10.222.0.50 () {54 vars in 921 bytes} [Fri May 27 03:26:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:26:20.042815 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a646a2-f50b-41af-94b9-906891c844fa None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:20.043293 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a646a2-f50b-41af-94b9-906891c844fa None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:20.043576 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a646a2-f50b-41af-94b9-906891c844fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:20.065544 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-42a646a2-f50b-41af-94b9-906891c844fa None None] MFA Rules not processed for user `4ee30c0d6d694ee8945800157a9fc5b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:20.066747 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-42a646a2-f50b-41af-94b9-906891c844fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:20.135880 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1092/2162] 10.222.0.50 () {58 vars in 1018 bytes} [Fri May 27 03:26:20 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:26:20.151043 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f82491-4dec-4d3a-b4c5-fc8e2eaf2cce None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:20.153410 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-d2f82491-4dec-4d3a-b4c5-fc8e2eaf2cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:20.167333 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f82491-4dec-4d3a-b4c5-fc8e2eaf2cce None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:20.171270 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c63f237-c481-4eee-91e4-4ae71d01ff1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:20.171910 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63f237-c481-4eee-91e4-4ae71d01ff1a service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:20.172187 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63f237-c481-4eee-91e4-4ae71d01ff1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:20.172465 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63f237-c481-4eee-91e4-4ae71d01ff1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:20.180062 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c63f237-c481-4eee-91e4-4ae71d01ff1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:20.181864 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c63f237-c481-4eee-91e4-4ae71d01ff1a service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:20.189921 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1071/2163] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:26:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:26:20.277430 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71d9b4-a50b-47d6-8a4f-44268eeedcf2 None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:20.277745 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71d9b4-a50b-47d6-8a4f-44268eeedcf2 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:20.278017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71d9b4-a50b-47d6-8a4f-44268eeedcf2 None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:20.279461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1093/2164] 10.222.0.50 () {54 vars in 921 bytes} [Fri May 27 03:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:26:20.285292 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4bd64-7403-4bd7-bbe5-52747da8fcb4 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:20.285599 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4bd64-7403-4bd7-bbe5-52747da8fcb4 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:20.285874 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4bd64-7403-4bd7-bbe5-52747da8fcb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:20.305814 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-bdd4bd64-7403-4bd7-bbe5-52747da8fcb4 None None] MFA Rules not processed for user `1a699988f08a4001b3b7d764f01f4fc5`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:20.307079 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-bdd4bd64-7403-4bd7-bbe5-52747da8fcb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:20.317805 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1072/2165] 10.222.0.50 () {58 vars in 1018 bytes} [Fri May 27 03:26:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:26:20.328734 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71d9b4-a50b-47d6-8a4f-44268eeedcf2 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:20.341657 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71d9b4-a50b-47d6-8a4f-44268eeedcf2 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:20.345695 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd914a32-8507-4615-858e-ea9ed2744bba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:20.346331 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd914a32-8507-4615-858e-ea9ed2744bba service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:20.346601 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd914a32-8507-4615-858e-ea9ed2744bba service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:20.346871 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd914a32-8507-4615-858e-ea9ed2744bba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:20.357604 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd914a32-8507-4615-858e-ea9ed2744bba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:20.359019 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd914a32-8507-4615-858e-ea9ed2744bba service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:20.369847 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1094/2166] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:26:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:26:23.235715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd4bd64-7403-4bd7-bbe5-52747da8fcb4 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:23.249830 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:23.328841 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd4bd64-7403-4bd7-bbe5-52747da8fcb4 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:23.333715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268cc796-11fe-46c5-b948-227b5e0b337e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:23.334607 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268cc796-11fe-46c5-b948-227b5e0b337e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:23.334888 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268cc796-11fe-46c5-b948-227b5e0b337e admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:23.335159 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268cc796-11fe-46c5-b948-227b5e0b337e admin admin] PATH_INFO: `/v3/users/feb7b7347fab4e6ea613c49c9be59eaf` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:23.336352 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268cc796-11fe-46c5-b948-227b5e0b337e admin admin] RBAC: Authorizing `identity:delete_user(user_id=feb7b7347fab4e6ea613c49c9be59eaf)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:23.338616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268cc796-11fe-46c5-b948-227b5e0b337e admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:23.392091 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-268cc796-11fe-46c5-b948-227b5e0b337e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'feb7b7347fab4e6ea613c49c9be59eaf'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:23.406453 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-268cc796-11fe-46c5-b948-227b5e0b337e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'feb7b7347fab4e6ea613c49c9be59eaf'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:23.417403 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-268cc796-11fe-46c5-b948-227b5e0b337e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'feb7b7347fab4e6ea613c49c9be59eaf'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:23.425537 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-268cc796-11fe-46c5-b948-227b5e0b337e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'feb7b7347fab4e6ea613c49c9be59eaf'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:23.426541 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-268cc796-11fe-46c5-b948-227b5e0b337e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'feb7b7347fab4e6ea613c49c9be59eaf'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:23.434376 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1073/2167] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:26:23 2022] DELETE /identity/v3/users/feb7b7347fab4e6ea613c49c9be59eaf => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:26:23.445225 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e8bc5-91a6-46b5-a79c-a6721e757aeb None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:23.445686 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e8bc5-91a6-46b5-a79c-a6721e757aeb None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:23.445897 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e8bc5-91a6-46b5-a79c-a6721e757aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:23.464264 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-ae6e8bc5-91a6-46b5-a79c-a6721e757aeb None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:23.465628 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-ae6e8bc5-91a6-46b5-a79c-a6721e757aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:23.526858 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1095/2168] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:26:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:26:23.538490 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268cc796-11fe-46c5-b948-227b5e0b337e admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:23.540045 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-268cc796-11fe-46c5-b948-227b5e0b337e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:23.604862 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268cc796-11fe-46c5-b948-227b5e0b337e admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:23.609801 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef910d00-3e64-44f5-aaf4-5d284722c8a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:23.610449 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef910d00-3e64-44f5-aaf4-5d284722c8a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:23.610730 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef910d00-3e64-44f5-aaf4-5d284722c8a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:23.610998 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef910d00-3e64-44f5-aaf4-5d284722c8a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:23.622010 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef910d00-3e64-44f5-aaf4-5d284722c8a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:23.623450 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef910d00-3e64-44f5-aaf4-5d284722c8a7 service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:23.633730 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1074/2169] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:26:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:26:23.744725 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bf0d3-9081-4980-8e02-f2cbbdf5aaa8 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:23.745668 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bf0d3-9081-4980-8e02-f2cbbdf5aaa8 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:23.746054 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bf0d3-9081-4980-8e02-f2cbbdf5aaa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:23.777573 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:23.806591 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-bb4bf0d3-9081-4980-8e02-f2cbbdf5aaa8 None None] MFA Rules not processed for user `86a18e313e4944748aef58d54bfe7b13`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:23.807869 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-bb4bf0d3-9081-4980-8e02-f2cbbdf5aaa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:23.876534 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1096/2170] 10.222.0.50 () {58 vars in 1009 bytes} [Fri May 27 03:26:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:26:23.889377 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef910d00-3e64-44f5-aaf4-5d284722c8a7 service neutron] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:23.890868 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-ef910d00-3e64-44f5-aaf4-5d284722c8a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:23.918972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4bf0d3-9081-4980-8e02-f2cbbdf5aaa8 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:23.920550 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-bb4bf0d3-9081-4980-8e02-f2cbbdf5aaa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:23.940176 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4bf0d3-9081-4980-8e02-f2cbbdf5aaa8 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:23.944984 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c720de2-70a2-4b61-8eb0-f15bd3120d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:23.945869 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c720de2-70a2-4b61-8eb0-f15bd3120d96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:23.946152 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c720de2-70a2-4b61-8eb0-f15bd3120d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:23.946429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c720de2-70a2-4b61-8eb0-f15bd3120d96 admin admin] PATH_INFO: `/v3/projects/76ca9458e9fb4201b02963e6d9be9db7` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:23.947825 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c720de2-70a2-4b61-8eb0-f15bd3120d96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76ca9458e9fb4201b02963e6d9be9db7)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:23.950485 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c720de2-70a2-4b61-8eb0-f15bd3120d96 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:23.961950 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:24.007781 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-4c720de2-70a2-4b61-8eb0-f15bd3120d96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76ca9458e9fb4201b02963e6d9be9db7', 'request_id': 'req-4c720de2-70a2-4b61-8eb0-f15bd3120d96'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:24.014932 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-4c720de2-70a2-4b61-8eb0-f15bd3120d96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76ca9458e9fb4201b02963e6d9be9db7', 'request_id': 'req-4c720de2-70a2-4b61-8eb0-f15bd3120d96'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:24.023543 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-4c720de2-70a2-4b61-8eb0-f15bd3120d96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76ca9458e9fb4201b02963e6d9be9db7', 'request_id': 'req-4c720de2-70a2-4b61-8eb0-f15bd3120d96'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:24.023756 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef910d00-3e64-44f5-aaf4-5d284722c8a7 service neutron] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:24.035922 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-4c720de2-70a2-4b61-8eb0-f15bd3120d96 admin admin] The token cache is being invalidate because project 76ca9458e9fb4201b02963e6d9be9db7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:26:24.036231 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-4c720de2-70a2-4b61-8eb0-f15bd3120d96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:24.036878 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-4c720de2-70a2-4b61-8eb0-f15bd3120d96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:26:24.038679 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1097/2171] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:26:23 2022] DELETE /identity/v3/projects/76ca9458e9fb4201b02963e6d9be9db7 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:26:24.043456 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaaf618-c31f-4f5e-877e-63b790b96382 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:24.043737 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaaf618-c31f-4f5e-877e-63b790b96382 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:24.044012 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaaf618-c31f-4f5e-877e-63b790b96382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:24.062999 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-1eaaf618-c31f-4f5e-877e-63b790b96382 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:24.064708 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-1eaaf618-c31f-4f5e-877e-63b790b96382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:24.084690 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47c7f8a-ad18-4ea7-b9a7-0cadac11164d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de53e2f0fba4e0fac70887067c1963d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:24.085396 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c7f8a-ad18-4ea7-b9a7-0cadac11164d service swift] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:24.085674 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c7f8a-ad18-4ea7-b9a7-0cadac11164d service swift] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:24.085950 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47c7f8a-ad18-4ea7-b9a7-0cadac11164d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:24.088058 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-d47c7f8a-ad18-4ea7-b9a7-0cadac11164d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:24.128293 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1098/2172] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:26:24.136185 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe52aac-d48c-4277-b07a-9da97753a7c6 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:24.136478 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe52aac-d48c-4277-b07a-9da97753a7c6 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:24.136749 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe52aac-d48c-4277-b07a-9da97753a7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:24.148314 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47c7f8a-ad18-4ea7-b9a7-0cadac11164d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:24.149989 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47c7f8a-ad18-4ea7-b9a7-0cadac11164d service swift] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:24.155109 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-2fe52aac-d48c-4277-b07a-9da97753a7c6 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:24.156305 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-2fe52aac-d48c-4277-b07a-9da97753a7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:24.158426 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1075/2173] 10.222.0.50 () {60 vars in 1384 bytes} [Fri May 27 03:26:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:26:24.164317 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1099/2174] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:26:24.168935 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47c7f8a-ad18-4ea7-b9a7-0cadac11164d service swift] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:24.179275 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47c7f8a-ad18-4ea7-b9a7-0cadac11164d service swift] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:24.184373 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb40b039-e476-4346-9403-9c91d688aafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:24.185183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb40b039-e476-4346-9403-9c91d688aafa admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:24.185460 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb40b039-e476-4346-9403-9c91d688aafa admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:24.185731 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb40b039-e476-4346-9403-9c91d688aafa admin admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:24.186455 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb40b039-e476-4346-9403-9c91d688aafa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:26:24.186756 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb40b039-e476-4346-9403-9c91d688aafa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:24.188354 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb40b039-e476-4346-9403-9c91d688aafa admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:24.201100 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1076/2175] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:26:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:26:24.206053 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe52aac-d48c-4277-b07a-9da97753a7c6 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:24.217097 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe52aac-d48c-4277-b07a-9da97753a7c6 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:24.222598 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6cb3a-3fbc-4581-ade6-dc315d6af72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:24.223342 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6cb3a-3fbc-4581-ade6-dc315d6af72f admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:24.223687 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6cb3a-3fbc-4581-ade6-dc315d6af72f admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:24.223959 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6cb3a-3fbc-4581-ade6-dc315d6af72f admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:24.224641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e6cb3a-3fbc-4581-ade6-dc315d6af72f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:24.226612 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e6cb3a-3fbc-4581-ade6-dc315d6af72f admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:24.231703 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:24.261360 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1100/2176] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:26:24 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:26:24.266753 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb40b039-e476-4346-9403-9c91d688aafa admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:24.285485 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb40b039-e476-4346-9403-9c91d688aafa admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:24.352842 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:24.377412 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8e6a2-e80a-445f-8537-64c65c344e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:24.378536 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8e6a2-e80a-445f-8537-64c65c344e0e admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:24.378765 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8e6a2-e80a-445f-8537-64c65c344e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:24.379199 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8e6a2-e80a-445f-8537-64c65c344e0e admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:24.380030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c8e6a2-e80a-445f-8537-64c65c344e0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:24.382765 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c8e6a2-e80a-445f-8537-64c65c344e0e admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:24.413179 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1077/2177] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:26:24 2022] POST /identity/v3/users => generated 288 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:26:24.417777 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6cb3a-3fbc-4581-ade6-dc315d6af72f admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:24.434078 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6cb3a-3fbc-4581-ade6-dc315d6af72f admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:24.442019 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4034ee6c-613f-44ac-ab4b-f80b4e837dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:24.442821 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4034ee6c-613f-44ac-ab4b-f80b4e837dbe admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:24.443279 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4034ee6c-613f-44ac-ab4b-f80b4e837dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:24.443612 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4034ee6c-613f-44ac-ab4b-f80b4e837dbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:24.446210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4034ee6c-613f-44ac-ab4b-f80b4e837dbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:24.448407 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4034ee6c-613f-44ac-ab4b-f80b4e837dbe admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:24.463079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1101/2178] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:26:24.463785 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8e6a2-e80a-445f-8537-64c65c344e0e admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:24.480163 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8e6a2-e80a-445f-8537-64c65c344e0e admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:24.490693 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9101ef24-ceed-4be1-aa79-a3931a8c59b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:24.491665 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101ef24-ceed-4be1-aa79-a3931a8c59b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:24.492099 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101ef24-ceed-4be1-aa79-a3931a8c59b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:24.492521 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101ef24-ceed-4be1-aa79-a3931a8c59b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:24.493764 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9101ef24-ceed-4be1-aa79-a3931a8c59b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:24.500284 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9101ef24-ceed-4be1-aa79-a3931a8c59b1 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:24.529504 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1078/2179] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:26:24.533623 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4034ee6c-613f-44ac-ab4b-f80b4e837dbe admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:24.552069 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4034ee6c-613f-44ac-ab4b-f80b4e837dbe admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:24.566515 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c0a9c-02e8-48ca-a8ba-1df1d7af0713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:24.567433 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c0a9c-02e8-48ca-a8ba-1df1d7af0713 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:24.567716 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c0a9c-02e8-48ca-a8ba-1df1d7af0713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:24.568037 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c0a9c-02e8-48ca-a8ba-1df1d7af0713 admin admin] PATH_INFO: `/v3/projects/121c8e84b0b34ac7870fd8f69d96dd8b/users/252f837ab93d40fe92f405a0fbffb82c/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:24.590617 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c0a9c-02e8-48ca-a8ba-1df1d7af0713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=121c8e84b0b34ac7870fd8f69d96dd8b, user_id=252f837ab93d40fe92f405a0fbffb82c, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:24.593836 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05c0a9c-02e8-48ca-a8ba-1df1d7af0713 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:24.614369 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1102/2180] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:26:24 2022] PUT /identity/v3/projects/121c8e84b0b34ac7870fd8f69d96dd8b/users/252f837ab93d40fe92f405a0fbffb82c/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:26:24.627251 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19580080-9373-4ee0-ae48-60a2f0c133c2 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:24.627657 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19580080-9373-4ee0-ae48-60a2f0c133c2 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:24.628756 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19580080-9373-4ee0-ae48-60a2f0c133c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:24.679461 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-19580080-9373-4ee0-ae48-60a2f0c133c2 None None] MFA Rules not processed for user `252f837ab93d40fe92f405a0fbffb82c`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:24.680761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-19580080-9373-4ee0-ae48-60a2f0c133c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:24.750222 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1079/2181] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:26:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:26:24.763833 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c0a9c-02e8-48ca-a8ba-1df1d7af0713 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:26:24.766051 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-d05c0a9c-02e8-48ca-a8ba-1df1d7af0713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:24.827112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c0a9c-02e8-48ca-a8ba-1df1d7af0713 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:26:24.830724 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc1d6f-7809-4765-9cad-333e60e03442 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:26:24.831369 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc1d6f-7809-4765-9cad-333e60e03442 service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:24.831645 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc1d6f-7809-4765-9cad-333e60e03442 service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:24.831915 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc1d6f-7809-4765-9cad-333e60e03442 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:24.840259 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cc1d6f-7809-4765-9cad-333e60e03442 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:26:24.841728 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cc1d6f-7809-4765-9cad-333e60e03442 service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:26:24.854347 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1103/2182] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:26:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:26:26.276921 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45431c6-67b8-4b28-869a-67d1e32b2105 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:26:26.277531 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45431c6-67b8-4b28-869a-67d1e32b2105 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:26:26.277531 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45431c6-67b8-4b28-869a-67d1e32b2105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:26:26.296185 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-f45431c6-67b8-4b28-869a-67d1e32b2105 None None] MFA Rules not processed for user `252f837ab93d40fe92f405a0fbffb82c`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:26:26.297537 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-f45431c6-67b8-4b28-869a-67d1e32b2105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:26:26.308122 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1080/2183] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:26:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:26:34.848078 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=46,keystone:INSERT=2 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:26:36.290254 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=50,keystone:INSERT=3 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:27:04.811563 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a38238-414d-416e-9df5-385a76c0a488 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:04.812168 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a38238-414d-416e-9df5-385a76c0a488 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:04.812168 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a38238-414d-416e-9df5-385a76c0a488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:04.836361 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-69a38238-414d-416e-9df5-385a76c0a488 None None] MFA Rules not processed for user `86a18e313e4944748aef58d54bfe7b13`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:04.837668 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-69a38238-414d-416e-9df5-385a76c0a488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:04.913448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1104/2184] 10.222.0.50 () {58 vars in 967 bytes} [Fri May 27 03:27:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:27:04.947962 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45431c6-67b8-4b28-869a-67d1e32b2105 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:04.950179 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-f45431c6-67b8-4b28-869a-67d1e32b2105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:05.047625 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45431c6-67b8-4b28-869a-67d1e32b2105 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:05.053913 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea93def8-42d4-4339-bca7-a3b6d764a7ec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de53e2f0fba4e0fac70887067c1963d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:05.054678 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93def8-42d4-4339-bca7-a3b6d764a7ec service swift] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:05.055028 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93def8-42d4-4339-bca7-a3b6d764a7ec service swift] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:05.055372 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea93def8-42d4-4339-bca7-a3b6d764a7ec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:05.064880 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea93def8-42d4-4339-bca7-a3b6d764a7ec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:05.067128 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea93def8-42d4-4339-bca7-a3b6d764a7ec service swift] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:05.076372 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1081/2185] 10.222.0.50 () {60 vars in 1384 bytes} [Fri May 27 03:27:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:27:06.538729 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97d32d-5792-4f35-87dd-d0c147355eb6 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:06.539403 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97d32d-5792-4f35-87dd-d0c147355eb6 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:06.539403 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97d32d-5792-4f35-87dd-d0c147355eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:06.561330 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-5a97d32d-5792-4f35-87dd-d0c147355eb6 None None] MFA Rules not processed for user `86a18e313e4944748aef58d54bfe7b13`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:06.562544 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-5a97d32d-5792-4f35-87dd-d0c147355eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:06.575240 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1105/2186] 10.222.0.50 () {58 vars in 1009 bytes} [Fri May 27 03:27:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:27:06.585342 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea93def8-42d4-4339-bca7-a3b6d764a7ec service swift] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:06.597266 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea93def8-42d4-4339-bca7-a3b6d764a7ec service swift] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:06.603933 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15e8fb-df23-43f3-bc58-249b9fe465a0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de53e2f0fba4e0fac70887067c1963d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:06.604644 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15e8fb-df23-43f3-bc58-249b9fe465a0 service swift] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:06.604856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15e8fb-df23-43f3-bc58-249b9fe465a0 service swift] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:06.605166 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15e8fb-df23-43f3-bc58-249b9fe465a0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:06.613913 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa15e8fb-df23-43f3-bc58-249b9fe465a0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:06.616008 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa15e8fb-df23-43f3-bc58-249b9fe465a0 service swift] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:06.624391 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1082/2187] 10.222.0.50 () {60 vars in 1384 bytes} [Fri May 27 03:27:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:27:07.208598 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662068fb-b53e-4e5e-b5d1-5dff8d2fb5b0 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:07.208827 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662068fb-b53e-4e5e-b5d1-5dff8d2fb5b0 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:07.209449 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662068fb-b53e-4e5e-b5d1-5dff8d2fb5b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:07.228491 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-662068fb-b53e-4e5e-b5d1-5dff8d2fb5b0 None None] MFA Rules not processed for user `86a18e313e4944748aef58d54bfe7b13`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:07.229721 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-662068fb-b53e-4e5e-b5d1-5dff8d2fb5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:07.237319 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1106/2188] 10.222.0.50 () {58 vars in 967 bytes} [Fri May 27 03:27:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:27:07.249645 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15e8fb-df23-43f3-bc58-249b9fe465a0 service swift] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:07.260686 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15e8fb-df23-43f3-bc58-249b9fe465a0 service swift] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:07.266230 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fcf5ca-3d4f-4e59-8d83-8deeb85d0be3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de53e2f0fba4e0fac70887067c1963d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:07.266868 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcf5ca-3d4f-4e59-8d83-8deeb85d0be3 service swift] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:07.267149 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcf5ca-3d4f-4e59-8d83-8deeb85d0be3 service swift] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:07.267427 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fcf5ca-3d4f-4e59-8d83-8deeb85d0be3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:07.276884 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fcf5ca-3d4f-4e59-8d83-8deeb85d0be3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:07.278580 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fcf5ca-3d4f-4e59-8d83-8deeb85d0be3 service swift] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:07.287418 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1083/2189] 10.222.0.50 () {60 vars in 1384 bytes} [Fri May 27 03:27:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:27:07.494165 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399446f7-03c5-4216-9672-38cd9ff80a8d None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:07.494455 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399446f7-03c5-4216-9672-38cd9ff80a8d None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:07.494793 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399446f7-03c5-4216-9672-38cd9ff80a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:07.513040 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-399446f7-03c5-4216-9672-38cd9ff80a8d None None] MFA Rules not processed for user `86a18e313e4944748aef58d54bfe7b13`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:07.514245 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-399446f7-03c5-4216-9672-38cd9ff80a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:07.521885 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1107/2190] 10.222.0.50 () {58 vars in 1009 bytes} [Fri May 27 03:27:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:27:07.533707 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fcf5ca-3d4f-4e59-8d83-8deeb85d0be3 service swift] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:07.544516 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fcf5ca-3d4f-4e59-8d83-8deeb85d0be3 service swift] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:07.549487 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28eb57-a624-4f1e-81a4-8ee78ae041c3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de53e2f0fba4e0fac70887067c1963d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:07.550129 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28eb57-a624-4f1e-81a4-8ee78ae041c3 service swift] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:07.550409 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28eb57-a624-4f1e-81a4-8ee78ae041c3 service swift] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:07.550712 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28eb57-a624-4f1e-81a4-8ee78ae041c3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:07.559114 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd28eb57-a624-4f1e-81a4-8ee78ae041c3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:07.560723 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd28eb57-a624-4f1e-81a4-8ee78ae041c3 service swift] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:07.569299 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1084/2191] 10.222.0.50 () {60 vars in 1384 bytes} [Fri May 27 03:27:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:27:07.901484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa4b9f-abb9-46d2-afc2-9fac890a7ddc None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:07.902004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa4b9f-abb9-46d2-afc2-9fac890a7ddc None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:07.902004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aa4b9f-abb9-46d2-afc2-9fac890a7ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:07.922244 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-b4aa4b9f-abb9-46d2-afc2-9fac890a7ddc None None] MFA Rules not processed for user `86a18e313e4944748aef58d54bfe7b13`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:07.923446 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-b4aa4b9f-abb9-46d2-afc2-9fac890a7ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:07.931594 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1108/2192] 10.222.0.50 () {58 vars in 967 bytes} [Fri May 27 03:27:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:27:07.944190 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28eb57-a624-4f1e-81a4-8ee78ae041c3 service swift] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:07.955222 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28eb57-a624-4f1e-81a4-8ee78ae041c3 service swift] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:07.960598 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86873a80-ba5b-475a-bd06-752a7577497d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de53e2f0fba4e0fac70887067c1963d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:07.961376 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86873a80-ba5b-475a-bd06-752a7577497d service swift] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:07.961723 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86873a80-ba5b-475a-bd06-752a7577497d service swift] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:07.962065 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86873a80-ba5b-475a-bd06-752a7577497d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:07.971158 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86873a80-ba5b-475a-bd06-752a7577497d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:07.972830 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86873a80-ba5b-475a-bd06-752a7577497d service swift] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:07.982086 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1085/2193] 10.222.0.50 () {60 vars in 1384 bytes} [Fri May 27 03:27:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:27:10.149397 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aa4b9f-abb9-46d2-afc2-9fac890a7ddc None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:10.215545 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aa4b9f-abb9-46d2-afc2-9fac890a7ddc None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:10.220267 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da4d7e-2a8d-4002-813b-cf76e7a826f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:10.221180 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da4d7e-2a8d-4002-813b-cf76e7a826f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:10.221458 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da4d7e-2a8d-4002-813b-cf76e7a826f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:10.221731 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da4d7e-2a8d-4002-813b-cf76e7a826f9 admin admin] PATH_INFO: `/v3/users/252f837ab93d40fe92f405a0fbffb82c` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:10.222941 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da4d7e-2a8d-4002-813b-cf76e7a826f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=252f837ab93d40fe92f405a0fbffb82c)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:10.224983 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da4d7e-2a8d-4002-813b-cf76e7a826f9 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:10.300389 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-e0da4d7e-2a8d-4002-813b-cf76e7a826f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '252f837ab93d40fe92f405a0fbffb82c'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:27:10.301174 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-e0da4d7e-2a8d-4002-813b-cf76e7a826f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '252f837ab93d40fe92f405a0fbffb82c'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:27:10.311359 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-e0da4d7e-2a8d-4002-813b-cf76e7a826f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '252f837ab93d40fe92f405a0fbffb82c'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:27:10.312354 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-e0da4d7e-2a8d-4002-813b-cf76e7a826f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '252f837ab93d40fe92f405a0fbffb82c'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:27:10.330101 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-e0da4d7e-2a8d-4002-813b-cf76e7a826f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '252f837ab93d40fe92f405a0fbffb82c'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:27:10.341273 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:27:10.348144 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1109/2194] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:27:10 2022] DELETE /identity/v3/users/252f837ab93d40fe92f405a0fbffb82c => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:27:10.352983 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac69f24f-fee2-4ceb-bdbb-1418e3a473d0 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:10.353495 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac69f24f-fee2-4ceb-bdbb-1418e3a473d0 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:10.353590 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac69f24f-fee2-4ceb-bdbb-1418e3a473d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:10.374132 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-ac69f24f-fee2-4ceb-bdbb-1418e3a473d0 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:10.375467 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-ac69f24f-fee2-4ceb-bdbb-1418e3a473d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:10.438053 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1086/2195] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:27:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:27:10.448635 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da4d7e-2a8d-4002-813b-cf76e7a826f9 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:10.450196 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-e0da4d7e-2a8d-4002-813b-cf76e7a826f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:10.510241 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da4d7e-2a8d-4002-813b-cf76e7a826f9 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:10.514668 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4937e445-8f37-4454-963e-8585d6401933 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:10.515298 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937e445-8f37-4454-963e-8585d6401933 service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:10.515571 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937e445-8f37-4454-963e-8585d6401933 service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:10.515842 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937e445-8f37-4454-963e-8585d6401933 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:10.526368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4937e445-8f37-4454-963e-8585d6401933 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:10.527820 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4937e445-8f37-4454-963e-8585d6401933 service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:10.538393 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1110/2196] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:27:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:27:10.721590 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac69f24f-fee2-4ceb-bdbb-1418e3a473d0 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:27:10.724178 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-ac69f24f-fee2-4ceb-bdbb-1418e3a473d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:10.741176 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac69f24f-fee2-4ceb-bdbb-1418e3a473d0 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:27:10.746232 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3afcf1e-7027-4c23-9e85-3bbb686c3646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:27:10.747097 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3afcf1e-7027-4c23-9e85-3bbb686c3646 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:10.747373 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3afcf1e-7027-4c23-9e85-3bbb686c3646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:10.747641 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3afcf1e-7027-4c23-9e85-3bbb686c3646 admin admin] PATH_INFO: `/v3/projects/121c8e84b0b34ac7870fd8f69d96dd8b` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:10.748874 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3afcf1e-7027-4c23-9e85-3bbb686c3646 admin admin] RBAC: Authorizing `identity:delete_project(project_id=121c8e84b0b34ac7870fd8f69d96dd8b)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:27:10.750834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3afcf1e-7027-4c23-9e85-3bbb686c3646 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:27:10.791872 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:27:10.804850 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-e3afcf1e-7027-4c23-9e85-3bbb686c3646 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '121c8e84b0b34ac7870fd8f69d96dd8b', 'request_id': 'req-e3afcf1e-7027-4c23-9e85-3bbb686c3646'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:27:10.811216 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-e3afcf1e-7027-4c23-9e85-3bbb686c3646 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '121c8e84b0b34ac7870fd8f69d96dd8b', 'request_id': 'req-e3afcf1e-7027-4c23-9e85-3bbb686c3646'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:27:10.817625 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-e3afcf1e-7027-4c23-9e85-3bbb686c3646 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '121c8e84b0b34ac7870fd8f69d96dd8b', 'request_id': 'req-e3afcf1e-7027-4c23-9e85-3bbb686c3646'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:27:10.832187 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-e3afcf1e-7027-4c23-9e85-3bbb686c3646 admin admin] The token cache is being invalidate because project 121c8e84b0b34ac7870fd8f69d96dd8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:27:10.832613 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-e3afcf1e-7027-4c23-9e85-3bbb686c3646 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:27:10.833477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-e3afcf1e-7027-4c23-9e85-3bbb686c3646 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:27:10.835073 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1087/2197] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:27:10 2022] DELETE /identity/v3/projects/121c8e84b0b34ac7870fd8f69d96dd8b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:27:19.282331 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd39cec-cde5-493e-90de-fcd66848607b None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:19.282839 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd39cec-cde5-493e-90de-fcd66848607b None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:19.282931 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd39cec-cde5-493e-90de-fcd66848607b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:19.304129 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-efd39cec-cde5-493e-90de-fcd66848607b None None] MFA Rules not processed for user `f0fbd32e5751496e8eaaf3bd03d8f833`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:19.305380 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-efd39cec-cde5-493e-90de-fcd66848607b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:19.363910 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1111/2198] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:27:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:27:20.829454 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=12,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:27:29.356767 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=52 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:27:50.251480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273ef9c-02e1-4afc-9873-f4265afa573a None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:50.252410 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273ef9c-02e1-4afc-9873-f4265afa573a None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:50.253103 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273ef9c-02e1-4afc-9873-f4265afa573a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:50.288054 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-b273ef9c-02e1-4afc-9873-f4265afa573a None None] MFA Rules not processed for user `468a46d55bd342739b9cd78bcd3a77fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:50.288054 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-b273ef9c-02e1-4afc-9873-f4265afa573a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:50.343069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1088/2199] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:27:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:27:51.661738 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260a521-2bdb-4c01-889f-b8eb6d82a078 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:27:51.662260 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260a521-2bdb-4c01-889f-b8eb6d82a078 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:27:51.662260 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3260a521-2bdb-4c01-889f-b8eb6d82a078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:27:51.681010 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-3260a521-2bdb-4c01-889f-b8eb6d82a078 None None] MFA Rules not processed for user `a094273e734342eca2bdf159fab8a8d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:27:51.682149 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-3260a521-2bdb-4c01-889f-b8eb6d82a078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:27:51.745017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1112/2200] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:27:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:28:00.336440 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=25 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:01.737932 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=25 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:07.608186 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ccfd1-3d97-40ed-ad7c-cea3e5ccb71b None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:07.609467 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ccfd1-3d97-40ed-ad7c-cea3e5ccb71b None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:07.609752 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ccfd1-3d97-40ed-ad7c-cea3e5ccb71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:07.627463 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-084ccfd1-3d97-40ed-ad7c-cea3e5ccb71b None None] MFA Rules not processed for user `dd9938d83dc84ae494310741e2cba2ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:07.628650 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-084ccfd1-3d97-40ed-ad7c-cea3e5ccb71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:07.685025 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1089/2201] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:28:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:28:13.005186 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3260a521-2bdb-4c01-889f-b8eb6d82a078 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:13.006881 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-3260a521-2bdb-4c01-889f-b8eb6d82a078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:13.072127 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3260a521-2bdb-4c01-889f-b8eb6d82a078 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:13.076565 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd0e858-8a37-4428-92e8-90955be713a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:13.077410 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd0e858-8a37-4428-92e8-90955be713a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:13.077738 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd0e858-8a37-4428-92e8-90955be713a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:13.077978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd0e858-8a37-4428-92e8-90955be713a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:13.080452 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-cfd0e858-8a37-4428-92e8-90955be713a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:13.170496 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd0e858-8a37-4428-92e8-90955be713a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:13.173238 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd0e858-8a37-4428-92e8-90955be713a2 service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:13.190691 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1113/2202] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:28:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:28:13.785455 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084ccfd1-3d97-40ed-ad7c-cea3e5ccb71b None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:13.787444 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-084ccfd1-3d97-40ed-ad7c-cea3e5ccb71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:13.800810 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084ccfd1-3d97-40ed-ad7c-cea3e5ccb71b None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:13.806119 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8514d25c-a0ad-45ae-afc8-b7753a23b106 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:13.807735 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8514d25c-a0ad-45ae-afc8-b7753a23b106 service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:13.808094 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8514d25c-a0ad-45ae-afc8-b7753a23b106 service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:13.808444 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8514d25c-a0ad-45ae-afc8-b7753a23b106 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:13.811851 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-8514d25c-a0ad-45ae-afc8-b7753a23b106 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:13.827044 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8514d25c-a0ad-45ae-afc8-b7753a23b106 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:13.828668 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8514d25c-a0ad-45ae-afc8-b7753a23b106 service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:13.841378 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1090/2203] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:28:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:28:18.114707 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b604f6f-6f4c-42be-9c5f-efc9aeca9da9 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:18.115252 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b604f6f-6f4c-42be-9c5f-efc9aeca9da9 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:18.115252 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b604f6f-6f4c-42be-9c5f-efc9aeca9da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:18.143170 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-6b604f6f-6f4c-42be-9c5f-efc9aeca9da9 None None] MFA Rules not processed for user `19457c759f0b4f5cbc530364af1f47d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:18.144398 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-6b604f6f-6f4c-42be-9c5f-efc9aeca9da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:18.209846 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1114/2204] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:28:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:28:18.610093 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8514d25c-a0ad-45ae-afc8-b7753a23b106 service neutron] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:18.618999 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8514d25c-a0ad-45ae-afc8-b7753a23b106 service neutron] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:18.622686 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036194e-d446-4cf5-ad51-d04d35210f1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:18.623328 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036194e-d446-4cf5-ad51-d04d35210f1c service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:18.623607 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036194e-d446-4cf5-ad51-d04d35210f1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:18.623882 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036194e-d446-4cf5-ad51-d04d35210f1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:18.629856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-d036194e-d446-4cf5-ad51-d04d35210f1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:18.641946 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d036194e-d446-4cf5-ad51-d04d35210f1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:18.643663 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d036194e-d446-4cf5-ad51-d04d35210f1c service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:18.653275 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1091/2205] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:28:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:28:19.155955 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b604f6f-6f4c-42be-9c5f-efc9aeca9da9 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:19.157736 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-6b604f6f-6f4c-42be-9c5f-efc9aeca9da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:19.224348 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:19.248518 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b604f6f-6f4c-42be-9c5f-efc9aeca9da9 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:19.254465 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd103b-6df7-4f91-9f73-fc747a480f29 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30963f2f464e4f54a8aea0175370a235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:19.255101 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd103b-6df7-4f91-9f73-fc747a480f29 service placement] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:19.255468 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd103b-6df7-4f91-9f73-fc747a480f29 service placement] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:19.255648 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecd103b-6df7-4f91-9f73-fc747a480f29 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:19.257560 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-cecd103b-6df7-4f91-9f73-fc747a480f29 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:19.272107 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecd103b-6df7-4f91-9f73-fc747a480f29 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:19.273959 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecd103b-6df7-4f91-9f73-fc747a480f29 service placement] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:19.285086 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1115/2206] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:28:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:28:22.316055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036194e-d446-4cf5-ad51-d04d35210f1c service neutron] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:22.321177 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-d036194e-d446-4cf5-ad51-d04d35210f1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:22.391132 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036194e-d446-4cf5-ad51-d04d35210f1c service neutron] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:22.396434 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129fb098-d6bd-4baa-81e4-541b7129bf2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:22.397359 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fb098-d6bd-4baa-81e4-541b7129bf2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:22.397812 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fb098-d6bd-4baa-81e4-541b7129bf2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:22.398245 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129fb098-d6bd-4baa-81e4-541b7129bf2c admin admin] PATH_INFO: `/v3/users/f0fbd32e5751496e8eaaf3bd03d8f833` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:22.400112 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129fb098-d6bd-4baa-81e4-541b7129bf2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0fbd32e5751496e8eaaf3bd03d8f833)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:22.402275 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129fb098-d6bd-4baa-81e4-541b7129bf2c admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:22.456106 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-129fb098-d6bd-4baa-81e4-541b7129bf2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0fbd32e5751496e8eaaf3bd03d8f833'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:28:22.472183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-129fb098-d6bd-4baa-81e4-541b7129bf2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0fbd32e5751496e8eaaf3bd03d8f833'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:28:22.484844 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-129fb098-d6bd-4baa-81e4-541b7129bf2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0fbd32e5751496e8eaaf3bd03d8f833'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:28:22.494034 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-129fb098-d6bd-4baa-81e4-541b7129bf2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0fbd32e5751496e8eaaf3bd03d8f833'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:28:22.494946 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-129fb098-d6bd-4baa-81e4-541b7129bf2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0fbd32e5751496e8eaaf3bd03d8f833'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:28:22.499703 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:22.509925 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1092/2207] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:28:22 2022] DELETE /identity/v3/users/f0fbd32e5751496e8eaaf3bd03d8f833 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:28:22.703521 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd103b-6df7-4f91-9f73-fc747a480f29 service placement] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:22.716062 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecd103b-6df7-4f91-9f73-fc747a480f29 service placement] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:22.775750 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526a5f44-4757-426e-83b7-7a715c8cd3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:22.776630 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a5f44-4757-426e-83b7-7a715c8cd3c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:22.776894 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a5f44-4757-426e-83b7-7a715c8cd3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:22.777192 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a5f44-4757-426e-83b7-7a715c8cd3c5 admin admin] PATH_INFO: `/v3/projects/4052d211d2a346d8be95d6ac7761defb` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:22.778351 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526a5f44-4757-426e-83b7-7a715c8cd3c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4052d211d2a346d8be95d6ac7761defb)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:22.780301 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526a5f44-4757-426e-83b7-7a715c8cd3c5 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:22.832115 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-526a5f44-4757-426e-83b7-7a715c8cd3c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4052d211d2a346d8be95d6ac7761defb', 'request_id': 'req-526a5f44-4757-426e-83b7-7a715c8cd3c5'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:28:22.838694 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-526a5f44-4757-426e-83b7-7a715c8cd3c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4052d211d2a346d8be95d6ac7761defb', 'request_id': 'req-526a5f44-4757-426e-83b7-7a715c8cd3c5'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:28:22.849056 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-526a5f44-4757-426e-83b7-7a715c8cd3c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4052d211d2a346d8be95d6ac7761defb', 'request_id': 'req-526a5f44-4757-426e-83b7-7a715c8cd3c5'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:28:22.860692 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-526a5f44-4757-426e-83b7-7a715c8cd3c5 admin admin] The token cache is being invalidate because project 4052d211d2a346d8be95d6ac7761defb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:28:22.861134 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-526a5f44-4757-426e-83b7-7a715c8cd3c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:28:22.861874 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-526a5f44-4757-426e-83b7-7a715c8cd3c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:28:22.863563 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1116/2208] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:28:22 2022] DELETE /identity/v3/projects/4052d211d2a346d8be95d6ac7761defb => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:28:22.868891 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afd56d-5a40-4f41-aaa1-9231f3804d0e None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:22.869208 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afd56d-5a40-4f41-aaa1-9231f3804d0e None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:22.869482 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afd56d-5a40-4f41-aaa1-9231f3804d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:22.888141 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-d6afd56d-5a40-4f41-aaa1-9231f3804d0e None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:22.889360 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-d6afd56d-5a40-4f41-aaa1-9231f3804d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:22.949959 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1093/2209] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:28:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:28:22.958038 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f47cc-7965-4411-9b64-4c417b040b66 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:22.958324 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f47cc-7965-4411-9b64-4c417b040b66 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:22.958598 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f47cc-7965-4411-9b64-4c417b040b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:22.977907 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-9d2f47cc-7965-4411-9b64-4c417b040b66 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:22.979112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-9d2f47cc-7965-4411-9b64-4c417b040b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:22.987757 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1117/2210] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:28:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:28:22.992105 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6afd56d-5a40-4f41-aaa1-9231f3804d0e None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:23.002322 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6afd56d-5a40-4f41-aaa1-9231f3804d0e None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:23.007966 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e466e5b-f58e-429f-8a63-e95b9dbbc4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:23.008622 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e466e5b-f58e-429f-8a63-e95b9dbbc4f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:23.009034 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e466e5b-f58e-429f-8a63-e95b9dbbc4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:23.009756 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e466e5b-f58e-429f-8a63-e95b9dbbc4f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:23.010496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e466e5b-f58e-429f-8a63-e95b9dbbc4f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:28:23.010800 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e466e5b-f58e-429f-8a63-e95b9dbbc4f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:23.012416 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e466e5b-f58e-429f-8a63-e95b9dbbc4f1 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:23.025178 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1094/2211] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:28:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:28:23.029552 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f47cc-7965-4411-9b64-4c417b040b66 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:23.039149 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f47cc-7965-4411-9b64-4c417b040b66 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:23.045244 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6a954-f2cc-46b5-899e-12351bd1982b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:23.045958 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6a954-f2cc-46b5-899e-12351bd1982b admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:23.046237 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6a954-f2cc-46b5-899e-12351bd1982b admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:23.046504 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6a954-f2cc-46b5-899e-12351bd1982b admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:23.047203 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb6a954-f2cc-46b5-899e-12351bd1982b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:23.049161 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb6a954-f2cc-46b5-899e-12351bd1982b admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:23.078461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1118/2212] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:28:23 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:28:23.082694 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e466e5b-f58e-429f-8a63-e95b9dbbc4f1 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:23.092938 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e466e5b-f58e-429f-8a63-e95b9dbbc4f1 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:23.157543 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3332b61a-d553-4568-a3f2-56632722a3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:23.158205 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332b61a-d553-4568-a3f2-56632722a3b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:23.158481 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332b61a-d553-4568-a3f2-56632722a3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:23.158749 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3332b61a-d553-4568-a3f2-56632722a3b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:23.159428 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3332b61a-d553-4568-a3f2-56632722a3b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:23.161418 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3332b61a-d553-4568-a3f2-56632722a3b6 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:23.184419 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1095/2213] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:28:23 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:28:23.188905 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6a954-f2cc-46b5-899e-12351bd1982b admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:23.200897 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6a954-f2cc-46b5-899e-12351bd1982b admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:23.207435 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489afc1c-898a-4568-a27a-e6ed8bb9ffb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:23.208201 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489afc1c-898a-4568-a27a-e6ed8bb9ffb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:23.208546 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489afc1c-898a-4568-a27a-e6ed8bb9ffb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:23.208819 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489afc1c-898a-4568-a27a-e6ed8bb9ffb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:23.209594 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489afc1c-898a-4568-a27a-e6ed8bb9ffb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:23.211213 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489afc1c-898a-4568-a27a-e6ed8bb9ffb8 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:23.223268 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1119/2214] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:28:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:28:23.227448 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3332b61a-d553-4568-a3f2-56632722a3b6 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:23.238457 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3332b61a-d553-4568-a3f2-56632722a3b6 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:23.243622 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd5b10c-9166-4fdb-a306-d9ef4da83295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:23.244303 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd5b10c-9166-4fdb-a306-d9ef4da83295 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:23.244540 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd5b10c-9166-4fdb-a306-d9ef4da83295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:23.244820 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd5b10c-9166-4fdb-a306-d9ef4da83295 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:23.245580 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd5b10c-9166-4fdb-a306-d9ef4da83295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:23.247182 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd5b10c-9166-4fdb-a306-d9ef4da83295 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:23.261395 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1096/2215] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:28:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:28:23.265707 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489afc1c-898a-4568-a27a-e6ed8bb9ffb8 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:23.278799 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489afc1c-898a-4568-a27a-e6ed8bb9ffb8 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:23.284828 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f00f251-0468-45d4-a047-6543e108174f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:23.285755 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f00f251-0468-45d4-a047-6543e108174f admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:23.286027 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f00f251-0468-45d4-a047-6543e108174f admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:23.286296 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f00f251-0468-45d4-a047-6543e108174f admin admin] PATH_INFO: `/v3/projects/f03723b938b845309929162c81adfe51/users/e38e8354a69642fba0c0949fe5f5dab3/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:23.303087 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f00f251-0468-45d4-a047-6543e108174f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f03723b938b845309929162c81adfe51, user_id=e38e8354a69642fba0c0949fe5f5dab3, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:23.305980 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f00f251-0468-45d4-a047-6543e108174f admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:23.324176 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1120/2216] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:28:23 2022] PUT /identity/v3/projects/f03723b938b845309929162c81adfe51/users/e38e8354a69642fba0c0949fe5f5dab3/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:28:23.325819 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4bac0c-c2dd-4220-a0dc-d2a6d24ca1e4 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:23.326626 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4bac0c-c2dd-4220-a0dc-d2a6d24ca1e4 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:23.326906 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4bac0c-c2dd-4220-a0dc-d2a6d24ca1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:23.347603 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-de4bac0c-c2dd-4220-a0dc-d2a6d24ca1e4 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:23.349223 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-de4bac0c-c2dd-4220-a0dc-d2a6d24ca1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:23.468799 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1097/2217] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:28:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:28:23.478656 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f00f251-0468-45d4-a047-6543e108174f admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:23.480697 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-3f00f251-0468-45d4-a047-6543e108174f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:23.486902 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:23.548595 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f00f251-0468-45d4-a047-6543e108174f admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:23.552380 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f595d35c-b974-459b-9668-48aeba016bde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:23.553062 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f595d35c-b974-459b-9668-48aeba016bde service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:23.553340 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f595d35c-b974-459b-9668-48aeba016bde service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:23.553608 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f595d35c-b974-459b-9668-48aeba016bde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:23.563535 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f595d35c-b974-459b-9668-48aeba016bde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:23.565267 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f595d35c-b974-459b-9668-48aeba016bde service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:23.575418 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1121/2218] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:28:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:28:28.246793 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4bac0c-c2dd-4220-a0dc-d2a6d24ca1e4 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:28.256515 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:28.271281 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4bac0c-c2dd-4220-a0dc-d2a6d24ca1e4 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:28.274799 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3537bce-64d5-4a82-a4a2-c0f84ca06c0d None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:28.275850 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3537bce-64d5-4a82-a4a2-c0f84ca06c0d None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:28.276316 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3537bce-64d5-4a82-a4a2-c0f84ca06c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:28.282672 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb04a0-d451-4046-ab36-5d14bc40bd90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:28.283357 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb04a0-d451-4046-ab36-5d14bc40bd90 service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:28.283632 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb04a0-d451-4046-ab36-5d14bc40bd90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:28.284255 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb04a0-d451-4046-ab36-5d14bc40bd90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:28.287321 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-52bb04a0-d451-4046-ab36-5d14bc40bd90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:28.299196 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bb04a0-d451-4046-ab36-5d14bc40bd90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:28.302017 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bb04a0-d451-4046-ab36-5d14bc40bd90 service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:28.304407 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-f3537bce-64d5-4a82-a4a2-c0f84ca06c0d None None] MFA Rules not processed for user `01be6ea7136e45aead352146193f05ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:28.305954 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-f3537bce-64d5-4a82-a4a2-c0f84ca06c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:28.329552 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1098/2219] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:28:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:28:28.436095 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1122/2220] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:28:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:28:30.262957 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95ed16-9514-490a-aaf7-99e8569896f9 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:30.264434 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95ed16-9514-490a-aaf7-99e8569896f9 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:30.264715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc95ed16-9514-490a-aaf7-99e8569896f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:30.297612 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-cc95ed16-9514-490a-aaf7-99e8569896f9 None None] MFA Rules not processed for user `e38e8354a69642fba0c0949fe5f5dab3`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:30.299162 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-cc95ed16-9514-490a-aaf7-99e8569896f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:30.377131 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1099/2221] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:28:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:28:30.380801 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3537bce-64d5-4a82-a4a2-c0f84ca06c0d None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:30.393443 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3537bce-64d5-4a82-a4a2-c0f84ca06c0d None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:30.398754 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3844ec50-625a-4b47-a04d-b5925cc43746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:30.399422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3844ec50-625a-4b47-a04d-b5925cc43746 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:30.399671 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3844ec50-625a-4b47-a04d-b5925cc43746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:30.399943 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3844ec50-625a-4b47-a04d-b5925cc43746 admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:30.400629 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3844ec50-625a-4b47-a04d-b5925cc43746 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:30.402621 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3844ec50-625a-4b47-a04d-b5925cc43746 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:30.432224 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1123/2222] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:28:30 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:28:30.436585 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95ed16-9514-490a-aaf7-99e8569896f9 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:30.448629 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc95ed16-9514-490a-aaf7-99e8569896f9 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:30.508622 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4ea95c-fac2-44f5-a011-0e2bfc68fddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:30.509756 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ea95c-fac2-44f5-a011-0e2bfc68fddb admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:30.510260 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ea95c-fac2-44f5-a011-0e2bfc68fddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:30.510652 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ea95c-fac2-44f5-a011-0e2bfc68fddb admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:30.511720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4ea95c-fac2-44f5-a011-0e2bfc68fddb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:30.513650 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4ea95c-fac2-44f5-a011-0e2bfc68fddb admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:30.550231 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1100/2223] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:28:30 2022] POST /identity/v3/users => generated 298 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:28:30.554289 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3844ec50-625a-4b47-a04d-b5925cc43746 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:30.568280 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3844ec50-625a-4b47-a04d-b5925cc43746 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:30.573211 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4425bd5-d98e-4bde-b44d-fd624a24318b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:30.574127 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4425bd5-d98e-4bde-b44d-fd624a24318b admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:30.574406 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4425bd5-d98e-4bde-b44d-fd624a24318b admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:30.574681 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4425bd5-d98e-4bde-b44d-fd624a24318b admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:30.575421 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4425bd5-d98e-4bde-b44d-fd624a24318b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:30.577445 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4425bd5-d98e-4bde-b44d-fd624a24318b admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:30.598362 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1124/2224] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:28:30 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:28:30.602698 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4ea95c-fac2-44f5-a011-0e2bfc68fddb admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:30.621260 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4ea95c-fac2-44f5-a011-0e2bfc68fddb admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:30.628436 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f9395a-d33f-4d60-81e9-c1e471c668dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:30.629487 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f9395a-d33f-4d60-81e9-c1e471c668dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:30.629867 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f9395a-d33f-4d60-81e9-c1e471c668dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:30.630036 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f9395a-d33f-4d60-81e9-c1e471c668dd admin admin] PATH_INFO: `/v3/projects/afbd09946f09447a92a186270aa81079/users/14aa67984a0d468d84c5f58c6b0b867f/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:30.651217 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f9395a-d33f-4d60-81e9-c1e471c668dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=afbd09946f09447a92a186270aa81079, user_id=14aa67984a0d468d84c5f58c6b0b867f, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:30.654702 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f9395a-d33f-4d60-81e9-c1e471c668dd admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:30.674804 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1101/2225] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:28:30 2022] PUT /identity/v3/projects/afbd09946f09447a92a186270aa81079/users/14aa67984a0d468d84c5f58c6b0b867f/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:28:30.678467 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4425bd5-d98e-4bde-b44d-fd624a24318b admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:30.691720 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4425bd5-d98e-4bde-b44d-fd624a24318b admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:30.764442 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25776eec-a770-434b-9818-dd75154b855a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:30.765125 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25776eec-a770-434b-9818-dd75154b855a admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:30.765356 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25776eec-a770-434b-9818-dd75154b855a admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:30.765798 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25776eec-a770-434b-9818-dd75154b855a admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:30.769926 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25776eec-a770-434b-9818-dd75154b855a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:30.780903 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25776eec-a770-434b-9818-dd75154b855a admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:30.806239 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1125/2226] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:28:30 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:28:30.810762 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f9395a-d33f-4d60-81e9-c1e471c668dd admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:30.821883 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f9395a-d33f-4d60-81e9-c1e471c668dd admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:30.828197 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b78811-0d3c-4436-abc5-3ef35cd3af93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:30.829360 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b78811-0d3c-4436-abc5-3ef35cd3af93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:30.829570 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b78811-0d3c-4436-abc5-3ef35cd3af93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:30.829955 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b78811-0d3c-4436-abc5-3ef35cd3af93 admin admin] PATH_INFO: `/v3/projects/afbd09946f09447a92a186270aa81079/users/14aa67984a0d468d84c5f58c6b0b867f/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:30.832230 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b78811-0d3c-4436-abc5-3ef35cd3af93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afbd09946f09447a92a186270aa81079, user_id=14aa67984a0d468d84c5f58c6b0b867f, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:30.835098 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b78811-0d3c-4436-abc5-3ef35cd3af93 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:30.856615 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1102/2227] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:28:30 2022] PUT /identity/v3/projects/afbd09946f09447a92a186270aa81079/users/14aa67984a0d468d84c5f58c6b0b867f/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:28:35.454287 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25776eec-a770-434b-9818-dd75154b855a admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:35.456429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-25776eec-a770-434b-9818-dd75154b855a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:35.473927 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:35.539887 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25776eec-a770-434b-9818-dd75154b855a admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:35.598175 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636c519-4c50-4151-bf28-e707a4ea6dd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:35.598836 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636c519-4c50-4151-bf28-e707a4ea6dd6 service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:35.599331 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636c519-4c50-4151-bf28-e707a4ea6dd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:35.600132 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636c519-4c50-4151-bf28-e707a4ea6dd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:35.602150 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-3636c519-4c50-4151-bf28-e707a4ea6dd6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:35.685021 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3636c519-4c50-4151-bf28-e707a4ea6dd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:35.686522 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3636c519-4c50-4151-bf28-e707a4ea6dd6 service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:35.697875 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1126/2228] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:28:35 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:28:37.075394 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041ed33-97e7-4a7c-a198-b0f808687e54 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:37.075936 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041ed33-97e7-4a7c-a198-b0f808687e54 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:37.075936 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d041ed33-97e7-4a7c-a198-b0f808687e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:37.108229 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-d041ed33-97e7-4a7c-a198-b0f808687e54 None None] MFA Rules not processed for user `14aa67984a0d468d84c5f58c6b0b867f`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:37.109617 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-d041ed33-97e7-4a7c-a198-b0f808687e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:37.134826 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:37.189717 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1103/2229] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:28:37 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:28:37.201785 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636c519-4c50-4151-bf28-e707a4ea6dd6 service cinder] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:37.205966 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-3636c519-4c50-4151-bf28-e707a4ea6dd6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:37.266772 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636c519-4c50-4151-bf28-e707a4ea6dd6 service cinder] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:37.270738 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a381ea83-facb-40e2-9f2a-ff3b2a28d884 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3c9c5c8d2e4376b1b70d974d4cb851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:37.271398 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a381ea83-facb-40e2-9f2a-ff3b2a28d884 service glance] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:37.271678 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a381ea83-facb-40e2-9f2a-ff3b2a28d884 service glance] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:37.271952 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a381ea83-facb-40e2-9f2a-ff3b2a28d884 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:37.313478 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:37.338796 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a381ea83-facb-40e2-9f2a-ff3b2a28d884 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:37.340304 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a381ea83-facb-40e2-9f2a-ff3b2a28d884 service glance] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:37.353210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1127/2230] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:28:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:28:37.495555 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a381ea83-facb-40e2-9f2a-ff3b2a28d884 service glance] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:37.496133 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31269d-d35f-4a9f-99cb-961e9bdd84c9 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:37.496613 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31269d-d35f-4a9f-99cb-961e9bdd84c9 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:37.497106 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c31269d-d35f-4a9f-99cb-961e9bdd84c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:37.509660 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-a381ea83-facb-40e2-9f2a-ff3b2a28d884 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:37.526835 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-8c31269d-d35f-4a9f-99cb-961e9bdd84c9 None None] MFA Rules not processed for user `86a18e313e4944748aef58d54bfe7b13`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:37.527977 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-8c31269d-d35f-4a9f-99cb-961e9bdd84c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:37.602040 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1104/2231] 10.222.0.50 () {58 vars in 1009 bytes} [Fri May 27 03:28:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:28:37.647273 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a381ea83-facb-40e2-9f2a-ff3b2a28d884 service glance] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:37.653778 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc499d8d-3c4e-45af-a8bb-39c12eb45d5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:37.654736 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc499d8d-3c4e-45af-a8bb-39c12eb45d5a service nova] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:37.655365 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc499d8d-3c4e-45af-a8bb-39c12eb45d5a service nova] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:37.655792 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc499d8d-3c4e-45af-a8bb-39c12eb45d5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:37.658497 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-dc499d8d-3c4e-45af-a8bb-39c12eb45d5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:37.682476 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c31269d-d35f-4a9f-99cb-961e9bdd84c9 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:37.683961 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-8c31269d-d35f-4a9f-99cb-961e9bdd84c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:37.733717 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc499d8d-3c4e-45af-a8bb-39c12eb45d5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:37.735982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc499d8d-3c4e-45af-a8bb-39c12eb45d5a service nova] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:37.747404 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1128/2232] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:28:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:28:37.773677 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c31269d-d35f-4a9f-99cb-961e9bdd84c9 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:37.789575 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e887b6e-6bb3-4149-978d-54664b839db1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de53e2f0fba4e0fac70887067c1963d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:37.790004 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e887b6e-6bb3-4149-978d-54664b839db1 service swift] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:37.790288 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e887b6e-6bb3-4149-978d-54664b839db1 service swift] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:37.790565 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e887b6e-6bb3-4149-978d-54664b839db1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:37.806644 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e887b6e-6bb3-4149-978d-54664b839db1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:37.808381 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e887b6e-6bb3-4149-978d-54664b839db1 service swift] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:37.819348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1105/2233] 10.222.0.50 () {60 vars in 1384 bytes} [Fri May 27 03:28:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 27 03:28:45.032028 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc499d8d-3c4e-45af-a8bb-39c12eb45d5a service nova] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:45.034195 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-dc499d8d-3c4e-45af-a8bb-39c12eb45d5a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:45.056580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc499d8d-3c4e-45af-a8bb-39c12eb45d5a service nova] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:45.065851 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da01c01-9f47-468e-a25b-bfec066bc00b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:45.067128 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da01c01-9f47-468e-a25b-bfec066bc00b service nova] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:45.067560 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da01c01-9f47-468e-a25b-bfec066bc00b service nova] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:45.067961 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da01c01-9f47-468e-a25b-bfec066bc00b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:45.070379 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-5da01c01-9f47-468e-a25b-bfec066bc00b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:45.101036 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da01c01-9f47-468e-a25b-bfec066bc00b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:45.106710 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da01c01-9f47-468e-a25b-bfec066bc00b service nova] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:45.139852 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1129/2234] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:28:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:28:47.817024 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=76 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:48.372896 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ce2f9-18d9-41db-9af0-239713547c7f None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:48.373322 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ce2f9-18d9-41db-9af0-239713547c7f None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:48.373726 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ce2f9-18d9-41db-9af0-239713547c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:48.408401 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-6d0ce2f9-18d9-41db-9af0-239713547c7f None None] MFA Rules not processed for user `e38e8354a69642fba0c0949fe5f5dab3`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:48.409780 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-6d0ce2f9-18d9-41db-9af0-239713547c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:48.421016 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1106/2235] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:28:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:28:48.674938 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da01c01-9f47-468e-a25b-bfec066bc00b service nova] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:48.676451 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-5da01c01-9f47-468e-a25b-bfec066bc00b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:48.736331 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da01c01-9f47-468e-a25b-bfec066bc00b service nova] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:48.740073 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b29ce89-0886-4819-9b08-f24e80b543aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:48.740762 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29ce89-0886-4819-9b08-f24e80b543aa service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:48.741081 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29ce89-0886-4819-9b08-f24e80b543aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:48.741359 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b29ce89-0886-4819-9b08-f24e80b543aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:48.743191 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-5b29ce89-0886-4819-9b08-f24e80b543aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:48.753757 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b29ce89-0886-4819-9b08-f24e80b543aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:48.755263 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b29ce89-0886-4819-9b08-f24e80b543aa service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:48.764292 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1130/2236] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:28:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:28:52.857993 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0ce2f9-18d9-41db-9af0-239713547c7f None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:52.871097 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0ce2f9-18d9-41db-9af0-239713547c7f None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:52.876893 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c14d62-c664-441f-91b0-8e9ae6cb40f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:52.877571 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c14d62-c664-441f-91b0-8e9ae6cb40f8 service nova] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:52.877877 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c14d62-c664-441f-91b0-8e9ae6cb40f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:52.878145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c14d62-c664-441f-91b0-8e9ae6cb40f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:52.879963 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-45c14d62-c664-441f-91b0-8e9ae6cb40f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:52.893075 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c14d62-c664-441f-91b0-8e9ae6cb40f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:52.894674 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c14d62-c664-441f-91b0-8e9ae6cb40f8 service nova] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:52.905874 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1107/2237] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:28:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:28:53.624869 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749bf4c-ef63-4fa2-a056-72736eda51f4 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:53.625534 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749bf4c-ef63-4fa2-a056-72736eda51f4 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:53.625646 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749bf4c-ef63-4fa2-a056-72736eda51f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:53.640470 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:28:53.664126 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-2749bf4c-ef63-4fa2-a056-72736eda51f4 None None] MFA Rules not processed for user `86a18e313e4944748aef58d54bfe7b13`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:28:53.665677 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-2749bf4c-ef63-4fa2-a056-72736eda51f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:28:53.686621 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1131/2238] 10.222.0.50 () {58 vars in 1009 bytes} [Fri May 27 03:28:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:28:53.700452 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c14d62-c664-441f-91b0-8e9ae6cb40f8 service nova] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:28:53.715860 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c14d62-c664-441f-91b0-8e9ae6cb40f8 service nova] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:28:53.725411 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7050e8-54b9-4a6b-b9a4-4ff340561157 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de53e2f0fba4e0fac70887067c1963d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:28:53.726467 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7050e8-54b9-4a6b-b9a4-4ff340561157 service swift] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:28:53.726841 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7050e8-54b9-4a6b-b9a4-4ff340561157 service swift] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:28:53.727204 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7050e8-54b9-4a6b-b9a4-4ff340561157 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:28:53.747873 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7050e8-54b9-4a6b-b9a4-4ff340561157 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:28:53.750034 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7050e8-54b9-4a6b-b9a4-4ff340561157 service swift] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:28:53.762944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1108/2239] 10.222.0.50 () {60 vars in 1384 bytes} [Fri May 27 03:28:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:29:00.498902 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2749bf4c-ef63-4fa2-a056-72736eda51f4 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:00.512992 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2749bf4c-ef63-4fa2-a056-72736eda51f4 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:00.518621 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b28d7-c946-475a-8f0f-03461e3661ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:00.519275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b28d7-c946-475a-8f0f-03461e3661ef service nova] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:00.519555 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b28d7-c946-475a-8f0f-03461e3661ef service nova] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:00.519829 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b28d7-c946-475a-8f0f-03461e3661ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:00.521620 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-b28b28d7-c946-475a-8f0f-03461e3661ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:00.580367 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28b28d7-c946-475a-8f0f-03461e3661ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:00.582012 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28b28d7-c946-475a-8f0f-03461e3661ef service nova] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:00.591654 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1132/2240] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:29:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:29:03.758660 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=15 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:10.586891 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=29 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:11.840805 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7050e8-54b9-4a6b-b9a4-4ff340561157 service swift] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:11.854854 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7050e8-54b9-4a6b-b9a4-4ff340561157 service swift] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:11.862529 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1db36-5432-48a6-a44b-b85e593c4760 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:11.863119 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1db36-5432-48a6-a44b-b85e593c4760 service nova] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:11.863479 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1db36-5432-48a6-a44b-b85e593c4760 service nova] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:11.863841 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1db36-5432-48a6-a44b-b85e593c4760 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:11.869176 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-29a1db36-5432-48a6-a44b-b85e593c4760 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:11.941992 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a1db36-5432-48a6-a44b-b85e593c4760 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:11.943930 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a1db36-5432-48a6-a44b-b85e593c4760 service nova] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:11.955548 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1109/2241] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:29:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:29:15.428362 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b28d7-c946-475a-8f0f-03461e3661ef service nova] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:15.438587 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b28d7-c946-475a-8f0f-03461e3661ef service nova] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:15.442597 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dee2875-76a7-49c7-9a80-ed5841dbf415 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:15.443270 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee2875-76a7-49c7-9a80-ed5841dbf415 service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:15.443598 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee2875-76a7-49c7-9a80-ed5841dbf415 service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:15.443913 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee2875-76a7-49c7-9a80-ed5841dbf415 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:15.445697 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-4dee2875-76a7-49c7-9a80-ed5841dbf415 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:15.522526 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dee2875-76a7-49c7-9a80-ed5841dbf415 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:15.523861 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dee2875-76a7-49c7-9a80-ed5841dbf415 service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:15.535576 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1133/2242] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:29:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:29:17.832821 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1db36-5432-48a6-a44b-b85e593c4760 service nova] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:17.846241 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1db36-5432-48a6-a44b-b85e593c4760 service nova] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:17.862008 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65065370-67b1-4d8d-936b-b1c08f6ac72a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:17.863478 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65065370-67b1-4d8d-936b-b1c08f6ac72a service nova] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:17.866259 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65065370-67b1-4d8d-936b-b1c08f6ac72a service nova] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:17.866786 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65065370-67b1-4d8d-936b-b1c08f6ac72a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:17.869975 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-65065370-67b1-4d8d-936b-b1c08f6ac72a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:17.890662 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65065370-67b1-4d8d-936b-b1c08f6ac72a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:17.892348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65065370-67b1-4d8d-936b-b1c08f6ac72a service nova] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:17.915261 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1110/2243] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:29:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:29:19.484389 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dee2875-76a7-49c7-9a80-ed5841dbf415 service neutron] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:19.487224 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-4dee2875-76a7-49c7-9a80-ed5841dbf415 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:19.589247 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dee2875-76a7-49c7-9a80-ed5841dbf415 service neutron] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:19.594469 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce14f4-138b-4538-95bd-c7257fc33f70 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30963f2f464e4f54a8aea0175370a235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:19.595138 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce14f4-138b-4538-95bd-c7257fc33f70 service placement] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:19.595415 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce14f4-138b-4538-95bd-c7257fc33f70 service placement] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:19.595874 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cce14f4-138b-4538-95bd-c7257fc33f70 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:19.597466 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65065370-67b1-4d8d-936b-b1c08f6ac72a service nova] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:19.597929 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-6cce14f4-138b-4538-95bd-c7257fc33f70 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:19.599056 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-65065370-67b1-4d8d-936b-b1c08f6ac72a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:19.610817 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cce14f4-138b-4538-95bd-c7257fc33f70 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:19.612342 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cce14f4-138b-4538-95bd-c7257fc33f70 service placement] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:19.617495 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65065370-67b1-4d8d-936b-b1c08f6ac72a service nova] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:19.624260 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1134/2244] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:29:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:29:19.624837 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9add0151-aebe-4170-8fd7-9cc366f285a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:19.626004 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add0151-aebe-4170-8fd7-9cc366f285a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:19.626328 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add0151-aebe-4170-8fd7-9cc366f285a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:19.626606 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add0151-aebe-4170-8fd7-9cc366f285a3 admin admin] PATH_INFO: `/v3/users/468a46d55bd342739b9cd78bcd3a77fc` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:19.627794 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9add0151-aebe-4170-8fd7-9cc366f285a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=468a46d55bd342739b9cd78bcd3a77fc)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:19.629776 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9add0151-aebe-4170-8fd7-9cc366f285a3 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:19.685841 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-9add0151-aebe-4170-8fd7-9cc366f285a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '468a46d55bd342739b9cd78bcd3a77fc'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:19.698101 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-9add0151-aebe-4170-8fd7-9cc366f285a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '468a46d55bd342739b9cd78bcd3a77fc'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:19.708761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-9add0151-aebe-4170-8fd7-9cc366f285a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '468a46d55bd342739b9cd78bcd3a77fc'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:19.716362 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-9add0151-aebe-4170-8fd7-9cc366f285a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '468a46d55bd342739b9cd78bcd3a77fc'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:19.717109 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-9add0151-aebe-4170-8fd7-9cc366f285a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '468a46d55bd342739b9cd78bcd3a77fc'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:19.724376 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1111/2245] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:29:19 2022] DELETE /identity/v3/users/468a46d55bd342739b9cd78bcd3a77fc => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:29:19.922898 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce14f4-138b-4538-95bd-c7257fc33f70 service placement] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:19.946842 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cce14f4-138b-4538-95bd-c7257fc33f70 service placement] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:20.069824 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68034f7a-6727-4537-adc5-32273ec9e234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:20.070709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68034f7a-6727-4537-adc5-32273ec9e234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:20.070996 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68034f7a-6727-4537-adc5-32273ec9e234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:20.071293 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68034f7a-6727-4537-adc5-32273ec9e234 admin admin] PATH_INFO: `/v3/projects/6db8193dcee944008c7b41889bfa67a3` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:20.072927 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68034f7a-6727-4537-adc5-32273ec9e234 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6db8193dcee944008c7b41889bfa67a3)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:20.075058 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68034f7a-6727-4537-adc5-32273ec9e234 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:20.116278 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:20.145105 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-68034f7a-6727-4537-adc5-32273ec9e234 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6db8193dcee944008c7b41889bfa67a3', 'request_id': 'req-68034f7a-6727-4537-adc5-32273ec9e234'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:20.153410 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-68034f7a-6727-4537-adc5-32273ec9e234 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6db8193dcee944008c7b41889bfa67a3', 'request_id': 'req-68034f7a-6727-4537-adc5-32273ec9e234'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:20.162888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-68034f7a-6727-4537-adc5-32273ec9e234 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6db8193dcee944008c7b41889bfa67a3', 'request_id': 'req-68034f7a-6727-4537-adc5-32273ec9e234'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:20.184257 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-68034f7a-6727-4537-adc5-32273ec9e234 admin admin] The token cache is being invalidate because project 6db8193dcee944008c7b41889bfa67a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:29:20.184689 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-68034f7a-6727-4537-adc5-32273ec9e234 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:20.185466 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-68034f7a-6727-4537-adc5-32273ec9e234 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:20.187380 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1135/2246] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:29:19 2022] DELETE /identity/v3/projects/6db8193dcee944008c7b41889bfa67a3 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:29:20.193061 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951a24c-cbed-4b8f-a43a-863961f123e7 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:20.193580 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951a24c-cbed-4b8f-a43a-863961f123e7 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:20.193694 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951a24c-cbed-4b8f-a43a-863961f123e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:20.215520 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-3951a24c-cbed-4b8f-a43a-863961f123e7 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:20.216745 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-3951a24c-cbed-4b8f-a43a-863961f123e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:20.280710 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:20.305021 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3b337-9585-462a-b4ff-650c0e9d004f None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:20.305382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3b337-9585-462a-b4ff-650c0e9d004f None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:20.305649 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1112/2247] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:29:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:29:20.306020 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3b337-9585-462a-b4ff-650c0e9d004f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:20.326741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-d7a3b337-9585-462a-b4ff-650c0e9d004f None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:20.328131 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-d7a3b337-9585-462a-b4ff-650c0e9d004f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:20.340188 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1136/2248] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:29:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:29:20.342052 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951a24c-cbed-4b8f-a43a-863961f123e7 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:20.352334 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951a24c-cbed-4b8f-a43a-863961f123e7 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:20.358035 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f52dd18-f27c-4638-8367-01be6433840b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:20.358723 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52dd18-f27c-4638-8367-01be6433840b admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:20.359006 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52dd18-f27c-4638-8367-01be6433840b admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:20.359281 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52dd18-f27c-4638-8367-01be6433840b admin admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:20.360023 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f52dd18-f27c-4638-8367-01be6433840b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:20.360330 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f52dd18-f27c-4638-8367-01be6433840b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:20.361988 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f52dd18-f27c-4638-8367-01be6433840b admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:20.379020 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1113/2249] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:29:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:20.380218 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a3b337-9585-462a-b4ff-650c0e9d004f None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:20.390385 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a3b337-9585-462a-b4ff-650c0e9d004f None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:20.395477 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d78f9-4a7d-4c56-a142-cffaef3156d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:20.396185 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d78f9-4a7d-4c56-a142-cffaef3156d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:20.396469 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d78f9-4a7d-4c56-a142-cffaef3156d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:20.396743 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d78f9-4a7d-4c56-a142-cffaef3156d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:20.397481 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60d78f9-4a7d-4c56-a142-cffaef3156d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:20.399414 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60d78f9-4a7d-4c56-a142-cffaef3156d9 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:20.444658 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1137/2250] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:29:20 2022] POST /identity/v3/projects => generated 363 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:29:20.449152 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f52dd18-f27c-4638-8367-01be6433840b admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:20.459897 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f52dd18-f27c-4638-8367-01be6433840b admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:20.538619 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afefe5bb-1289-43a0-8562-c864b62475ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:20.540176 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afefe5bb-1289-43a0-8562-c864b62475ee admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:20.540855 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afefe5bb-1289-43a0-8562-c864b62475ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:20.541384 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afefe5bb-1289-43a0-8562-c864b62475ee admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:20.543074 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afefe5bb-1289-43a0-8562-c864b62475ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:20.545827 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afefe5bb-1289-43a0-8562-c864b62475ee admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:20.581077 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1114/2251] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:29:20 2022] POST /identity/v3/users => generated 282 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:29:20.585445 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d78f9-4a7d-4c56-a142-cffaef3156d9 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:20.613206 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d78f9-4a7d-4c56-a142-cffaef3156d9 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:20.620524 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12577c-b543-436e-986f-f13e900f844c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:20.622164 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12577c-b543-436e-986f-f13e900f844c admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:20.622768 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12577c-b543-436e-986f-f13e900f844c admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:20.623221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12577c-b543-436e-986f-f13e900f844c admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:20.624139 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f12577c-b543-436e-986f-f13e900f844c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:20.626201 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f12577c-b543-436e-986f-f13e900f844c admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:20.643353 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1138/2252] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:29:20 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:29:20.650767 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afefe5bb-1289-43a0-8562-c864b62475ee admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:20.658831 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afefe5bb-1289-43a0-8562-c864b62475ee admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:20.663578 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663638e-796d-4e25-9d75-4ba51df0f2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:20.664256 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663638e-796d-4e25-9d75-4ba51df0f2a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:20.664537 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663638e-796d-4e25-9d75-4ba51df0f2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:20.664813 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663638e-796d-4e25-9d75-4ba51df0f2a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:20.665717 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b663638e-796d-4e25-9d75-4ba51df0f2a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:20.668079 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b663638e-796d-4e25-9d75-4ba51df0f2a8 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:20.684770 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1115/2253] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:29:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:29:20.686025 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12577c-b543-436e-986f-f13e900f844c admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:20.697214 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12577c-b543-436e-986f-f13e900f844c admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:20.702003 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a461b976-81f1-4f87-b1cf-37b0dd4a55b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:20.702904 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461b976-81f1-4f87-b1cf-37b0dd4a55b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:20.703182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461b976-81f1-4f87-b1cf-37b0dd4a55b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:20.703458 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a461b976-81f1-4f87-b1cf-37b0dd4a55b2 admin admin] PATH_INFO: `/v3/projects/ba2f445c35674bc39734f54d78a44e8d/users/24fa53d6dcc94f20b7943a0fd4d4d752/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:20.721162 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a461b976-81f1-4f87-b1cf-37b0dd4a55b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba2f445c35674bc39734f54d78a44e8d, user_id=24fa53d6dcc94f20b7943a0fd4d4d752, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:20.724842 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a461b976-81f1-4f87-b1cf-37b0dd4a55b2 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:20.744589 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1139/2254] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:29:20 2022] PUT /identity/v3/projects/ba2f445c35674bc39734f54d78a44e8d/users/24fa53d6dcc94f20b7943a0fd4d4d752/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:29:20.754966 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01472fdf-0ff7-4884-b78a-3570a7bdd170 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:20.755253 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01472fdf-0ff7-4884-b78a-3570a7bdd170 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:20.755533 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01472fdf-0ff7-4884-b78a-3570a7bdd170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:20.785408 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-01472fdf-0ff7-4884-b78a-3570a7bdd170 None None] MFA Rules not processed for user `24fa53d6dcc94f20b7943a0fd4d4d752`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:20.786570 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-01472fdf-0ff7-4884-b78a-3570a7bdd170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:20.876702 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1116/2255] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:29:20 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:29:20.881575 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a461b976-81f1-4f87-b1cf-37b0dd4a55b2 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:20.955619 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a461b976-81f1-4f87-b1cf-37b0dd4a55b2 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:20.960495 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69ddf6-6e0b-4096-bd40-8fc984bb9b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:20.961266 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69ddf6-6e0b-4096-bd40-8fc984bb9b23 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:20.961525 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69ddf6-6e0b-4096-bd40-8fc984bb9b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:20.961801 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69ddf6-6e0b-4096-bd40-8fc984bb9b23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:20.962554 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f69ddf6-6e0b-4096-bd40-8fc984bb9b23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:20.964778 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f69ddf6-6e0b-4096-bd40-8fc984bb9b23 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:20.999505 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1140/2256] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:29:20 2022] POST /identity/v3/projects => generated 365 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:29:21.000783 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01472fdf-0ff7-4884-b78a-3570a7bdd170 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:21.015287 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01472fdf-0ff7-4884-b78a-3570a7bdd170 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:21.079572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:21.098044 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69154434-1178-472e-895a-9c97e780504b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:21.099436 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69154434-1178-472e-895a-9c97e780504b admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:21.099793 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69154434-1178-472e-895a-9c97e780504b admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:21.100420 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69154434-1178-472e-895a-9c97e780504b admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:21.101357 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69154434-1178-472e-895a-9c97e780504b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:21.104371 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69154434-1178-472e-895a-9c97e780504b admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:21.137970 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1117/2257] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:29:20 2022] POST /identity/v3/users => generated 289 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:29:21.139328 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69ddf6-6e0b-4096-bd40-8fc984bb9b23 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:21.151108 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69ddf6-6e0b-4096-bd40-8fc984bb9b23 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:21.159055 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ca790b-02e8-4215-a658-7b5311dc8500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:21.159770 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca790b-02e8-4215-a658-7b5311dc8500 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:21.160549 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca790b-02e8-4215-a658-7b5311dc8500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:21.160549 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca790b-02e8-4215-a658-7b5311dc8500 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:21.161670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ca790b-02e8-4215-a658-7b5311dc8500 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:21.165018 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ca790b-02e8-4215-a658-7b5311dc8500 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:21.178956 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1141/2258] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:29:21.183621 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69154434-1178-472e-895a-9c97e780504b admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:21.196542 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69154434-1178-472e-895a-9c97e780504b admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:21.202464 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24591a42-04ed-4d0a-aa8e-7b0ffda5b868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:21.203405 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24591a42-04ed-4d0a-aa8e-7b0ffda5b868 admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:21.203640 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24591a42-04ed-4d0a-aa8e-7b0ffda5b868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:21.203915 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24591a42-04ed-4d0a-aa8e-7b0ffda5b868 admin admin] PATH_INFO: `/v3/projects/57243a254e864281a3d77c2b8d160f1b/users/0191e878bdd84257a24c2a52d14197d5/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:21.222440 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24591a42-04ed-4d0a-aa8e-7b0ffda5b868 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57243a254e864281a3d77c2b8d160f1b, user_id=0191e878bdd84257a24c2a52d14197d5, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:21.227325 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24591a42-04ed-4d0a-aa8e-7b0ffda5b868 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:21.252721 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1118/2259] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:29:21 2022] PUT /identity/v3/projects/57243a254e864281a3d77c2b8d160f1b/users/0191e878bdd84257a24c2a52d14197d5/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:29:21.254433 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ca790b-02e8-4215-a658-7b5311dc8500 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:21.268204 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ca790b-02e8-4215-a658-7b5311dc8500 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:21.316656 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:21.366825 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2674f-8dab-479d-b760-3e0051476202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:21.367959 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2674f-8dab-479d-b760-3e0051476202 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:21.367959 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2674f-8dab-479d-b760-3e0051476202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:21.368188 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2674f-8dab-479d-b760-3e0051476202 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:21.368912 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b2674f-8dab-479d-b760-3e0051476202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:21.370625 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b2674f-8dab-479d-b760-3e0051476202 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:21.381274 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1142/2260] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:29:21.386053 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24591a42-04ed-4d0a-aa8e-7b0ffda5b868 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:21.401227 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24591a42-04ed-4d0a-aa8e-7b0ffda5b868 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:21.406296 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bfc4d5-0ac5-4a6a-af82-0957099a2dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:21.407267 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bfc4d5-0ac5-4a6a-af82-0957099a2dc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:21.407633 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bfc4d5-0ac5-4a6a-af82-0957099a2dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:21.407979 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bfc4d5-0ac5-4a6a-af82-0957099a2dc3 admin admin] PATH_INFO: `/v3/projects/57243a254e864281a3d77c2b8d160f1b/users/0191e878bdd84257a24c2a52d14197d5/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:21.410146 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bfc4d5-0ac5-4a6a-af82-0957099a2dc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57243a254e864281a3d77c2b8d160f1b, user_id=0191e878bdd84257a24c2a52d14197d5, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:21.412907 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bfc4d5-0ac5-4a6a-af82-0957099a2dc3 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:21.424727 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1119/2261] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:29:21 2022] PUT /identity/v3/projects/57243a254e864281a3d77c2b8d160f1b/users/0191e878bdd84257a24c2a52d14197d5/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:29:21.437253 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64f3b4-84d2-4ac8-bafb-35070789bd49 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:21.437679 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64f3b4-84d2-4ac8-bafb-35070789bd49 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:21.437958 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c64f3b4-84d2-4ac8-bafb-35070789bd49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:21.466763 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-7c64f3b4-84d2-4ac8-bafb-35070789bd49 None None] MFA Rules not processed for user `0191e878bdd84257a24c2a52d14197d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:21.468009 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-7c64f3b4-84d2-4ac8-bafb-35070789bd49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:21.544143 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1143/2262] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:29:21 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:29:21.557271 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bfc4d5-0ac5-4a6a-af82-0957099a2dc3 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:21.558993 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-08bfc4d5-0ac5-4a6a-af82-0957099a2dc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:21.622652 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bfc4d5-0ac5-4a6a-af82-0957099a2dc3 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:21.626607 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73810c14-0d95-4d4d-bb77-67572b2d4a33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:21.627294 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73810c14-0d95-4d4d-bb77-67572b2d4a33 service cinder] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:21.627576 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73810c14-0d95-4d4d-bb77-67572b2d4a33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:21.627852 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73810c14-0d95-4d4d-bb77-67572b2d4a33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:21.638171 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73810c14-0d95-4d4d-bb77-67572b2d4a33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:21.639719 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73810c14-0d95-4d4d-bb77-67572b2d4a33 service cinder] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:21.649669 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1120/2263] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:29:21 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:29:21.727993 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c64f3b4-84d2-4ac8-bafb-35070789bd49 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:21.738783 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c64f3b4-84d2-4ac8-bafb-35070789bd49 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:21.743082 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91793211-e98f-4959-888b-accb23a9c8a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:21.743773 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91793211-e98f-4959-888b-accb23a9c8a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:21.743773 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91793211-e98f-4959-888b-accb23a9c8a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:21.744074 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91793211-e98f-4959-888b-accb23a9c8a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:21.819954 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91793211-e98f-4959-888b-accb23a9c8a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:21.821472 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91793211-e98f-4959-888b-accb23a9c8a3 service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:21.831307 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1144/2264] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:29:21 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:29:22.917128 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a91dd-4114-4b93-8233-b564e209675c None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:22.917613 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a91dd-4114-4b93-8233-b564e209675c None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:22.917734 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a91dd-4114-4b93-8233-b564e209675c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:22.940015 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-af3a91dd-4114-4b93-8233-b564e209675c None None] MFA Rules not processed for user `a094273e734342eca2bdf159fab8a8d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:22.941316 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-af3a91dd-4114-4b93-8233-b564e209675c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:23.016630 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1121/2265] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:29:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:29:31.826962 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=80 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:31.985918 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91793211-e98f-4959-888b-accb23a9c8a3 service cinder] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:31.987585 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-91793211-e98f-4959-888b-accb23a9c8a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:32.052888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91793211-e98f-4959-888b-accb23a9c8a3 service cinder] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:32.056977 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6f08b1-84f4-4b1a-a429-d4ecc8943a17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:32.057633 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f08b1-84f4-4b1a-a429-d4ecc8943a17 service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:32.057913 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f08b1-84f4-4b1a-a429-d4ecc8943a17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:32.058182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6f08b1-84f4-4b1a-a429-d4ecc8943a17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:32.060009 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-2d6f08b1-84f4-4b1a-a429-d4ecc8943a17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:32.134345 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6f08b1-84f4-4b1a-a429-d4ecc8943a17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:32.135964 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6f08b1-84f4-4b1a-a429-d4ecc8943a17 service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:32.146807 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1145/2266] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:29:31 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:29:32.268405 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a91dd-4114-4b93-8233-b564e209675c None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:32.333372 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:32.347069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a91dd-4114-4b93-8233-b564e209675c None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:32.353479 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429b956a-d97a-4617-88ec-a6c8e7f8a7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:32.354539 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429b956a-d97a-4617-88ec-a6c8e7f8a7c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:32.354906 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429b956a-d97a-4617-88ec-a6c8e7f8a7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:32.355093 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429b956a-d97a-4617-88ec-a6c8e7f8a7c6 admin admin] PATH_INFO: `/v3/users/24fa53d6dcc94f20b7943a0fd4d4d752` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:32.356298 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429b956a-d97a-4617-88ec-a6c8e7f8a7c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24fa53d6dcc94f20b7943a0fd4d4d752)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:32.358726 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429b956a-d97a-4617-88ec-a6c8e7f8a7c6 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:32.425608 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-429b956a-d97a-4617-88ec-a6c8e7f8a7c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24fa53d6dcc94f20b7943a0fd4d4d752'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:32.439115 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-429b956a-d97a-4617-88ec-a6c8e7f8a7c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24fa53d6dcc94f20b7943a0fd4d4d752'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:32.448864 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-429b956a-d97a-4617-88ec-a6c8e7f8a7c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24fa53d6dcc94f20b7943a0fd4d4d752'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:32.458307 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-429b956a-d97a-4617-88ec-a6c8e7f8a7c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24fa53d6dcc94f20b7943a0fd4d4d752'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:32.458992 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-429b956a-d97a-4617-88ec-a6c8e7f8a7c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24fa53d6dcc94f20b7943a0fd4d4d752'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:32.465819 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1122/2267] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:29:32 2022] DELETE /identity/v3/users/24fa53d6dcc94f20b7943a0fd4d4d752 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:29:32.470767 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9976e-55a6-4c1f-9870-73d5b037ed5e None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:32.471052 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9976e-55a6-4c1f-9870-73d5b037ed5e None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:32.471329 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea9976e-55a6-4c1f-9870-73d5b037ed5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:32.489464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-3ea9976e-55a6-4c1f-9870-73d5b037ed5e None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:32.490683 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-3ea9976e-55a6-4c1f-9870-73d5b037ed5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:32.555297 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1146/2268] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:29:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:29:32.565904 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429b956a-d97a-4617-88ec-a6c8e7f8a7c6 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:32.567666 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-429b956a-d97a-4617-88ec-a6c8e7f8a7c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:32.627467 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429b956a-d97a-4617-88ec-a6c8e7f8a7c6 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:32.631061 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40efc8a-7539-4800-86d6-f1a90750f969 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:32.631916 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40efc8a-7539-4800-86d6-f1a90750f969 service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:32.632185 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40efc8a-7539-4800-86d6-f1a90750f969 service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:32.632450 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40efc8a-7539-4800-86d6-f1a90750f969 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:32.642496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40efc8a-7539-4800-86d6-f1a90750f969 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:32.643992 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40efc8a-7539-4800-86d6-f1a90750f969 service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:32.653677 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1123/2269] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:29:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:29:32.941137 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9976e-55a6-4c1f-9870-73d5b037ed5e None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:32.942625 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-3ea9976e-55a6-4c1f-9870-73d5b037ed5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:32.955930 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea9976e-55a6-4c1f-9870-73d5b037ed5e None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:32.960972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014539da-dcb2-4f38-84b9-04367ad81c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:32.961870 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014539da-dcb2-4f38-84b9-04367ad81c1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:32.962148 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014539da-dcb2-4f38-84b9-04367ad81c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:32.962433 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014539da-dcb2-4f38-84b9-04367ad81c1c admin admin] PATH_INFO: `/v3/projects/ba2f445c35674bc39734f54d78a44e8d` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:32.963619 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014539da-dcb2-4f38-84b9-04367ad81c1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba2f445c35674bc39734f54d78a44e8d)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:32.965600 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014539da-dcb2-4f38-84b9-04367ad81c1c admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:33.030930 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:33.034400 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-014539da-dcb2-4f38-84b9-04367ad81c1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba2f445c35674bc39734f54d78a44e8d', 'request_id': 'req-014539da-dcb2-4f38-84b9-04367ad81c1c'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:33.042688 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-014539da-dcb2-4f38-84b9-04367ad81c1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba2f445c35674bc39734f54d78a44e8d', 'request_id': 'req-014539da-dcb2-4f38-84b9-04367ad81c1c'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:33.053332 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-014539da-dcb2-4f38-84b9-04367ad81c1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba2f445c35674bc39734f54d78a44e8d', 'request_id': 'req-014539da-dcb2-4f38-84b9-04367ad81c1c'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:33.065013 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-014539da-dcb2-4f38-84b9-04367ad81c1c admin admin] The token cache is being invalidate because project ba2f445c35674bc39734f54d78a44e8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:29:33.065371 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-014539da-dcb2-4f38-84b9-04367ad81c1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:33.066024 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-014539da-dcb2-4f38-84b9-04367ad81c1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:33.067633 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1147/2270] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:29:32 2022] DELETE /identity/v3/projects/ba2f445c35674bc39734f54d78a44e8d => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:29:33.071837 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40efc8a-7539-4800-86d6-f1a90750f969 service neutron] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:33.081800 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40efc8a-7539-4800-86d6-f1a90750f969 service neutron] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:33.148214 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e688f4-cbca-407d-ad4a-412ba2ac2ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:33.149340 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e688f4-cbca-407d-ad4a-412ba2ac2ea7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:33.149622 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e688f4-cbca-407d-ad4a-412ba2ac2ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:33.149895 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e688f4-cbca-407d-ad4a-412ba2ac2ea7 admin admin] PATH_INFO: `/v3/users/0191e878bdd84257a24c2a52d14197d5` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:33.151198 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e688f4-cbca-407d-ad4a-412ba2ac2ea7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0191e878bdd84257a24c2a52d14197d5)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:33.153350 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e688f4-cbca-407d-ad4a-412ba2ac2ea7 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:33.177637 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:33.227590 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-a0e688f4-cbca-407d-ad4a-412ba2ac2ea7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0191e878bdd84257a24c2a52d14197d5'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:33.245045 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-a0e688f4-cbca-407d-ad4a-412ba2ac2ea7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0191e878bdd84257a24c2a52d14197d5'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:33.255572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-a0e688f4-cbca-407d-ad4a-412ba2ac2ea7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0191e878bdd84257a24c2a52d14197d5'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:33.264565 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-a0e688f4-cbca-407d-ad4a-412ba2ac2ea7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0191e878bdd84257a24c2a52d14197d5'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:33.265332 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-a0e688f4-cbca-407d-ad4a-412ba2ac2ea7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0191e878bdd84257a24c2a52d14197d5'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:33.273549 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1124/2271] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:29:33 2022] DELETE /identity/v3/users/0191e878bdd84257a24c2a52d14197d5 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:29:33.584588 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014539da-dcb2-4f38-84b9-04367ad81c1c admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:33.586078 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-014539da-dcb2-4f38-84b9-04367ad81c1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:33.601936 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014539da-dcb2-4f38-84b9-04367ad81c1c admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:33.683000 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8be46-8829-45b0-ab1d-fdbd3935916f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:33.683845 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8be46-8829-45b0-ab1d-fdbd3935916f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:33.684319 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8be46-8829-45b0-ab1d-fdbd3935916f admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:33.684829 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8be46-8829-45b0-ab1d-fdbd3935916f admin admin] PATH_INFO: `/v3/projects/57243a254e864281a3d77c2b8d160f1b` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:33.686742 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b8be46-8829-45b0-ab1d-fdbd3935916f admin admin] RBAC: Authorizing `identity:delete_project(project_id=57243a254e864281a3d77c2b8d160f1b)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:33.689415 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b8be46-8829-45b0-ab1d-fdbd3935916f admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:33.754107 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-09b8be46-8829-45b0-ab1d-fdbd3935916f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57243a254e864281a3d77c2b8d160f1b', 'request_id': 'req-09b8be46-8829-45b0-ab1d-fdbd3935916f'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:33.763913 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-09b8be46-8829-45b0-ab1d-fdbd3935916f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57243a254e864281a3d77c2b8d160f1b', 'request_id': 'req-09b8be46-8829-45b0-ab1d-fdbd3935916f'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:33.780252 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-09b8be46-8829-45b0-ab1d-fdbd3935916f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57243a254e864281a3d77c2b8d160f1b', 'request_id': 'req-09b8be46-8829-45b0-ab1d-fdbd3935916f'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:33.806722 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-09b8be46-8829-45b0-ab1d-fdbd3935916f admin admin] The token cache is being invalidate because project 57243a254e864281a3d77c2b8d160f1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:29:33.807160 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-09b8be46-8829-45b0-ab1d-fdbd3935916f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:33.808615 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-09b8be46-8829-45b0-ab1d-fdbd3935916f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:29:33.814785 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1148/2272] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:29:33 2022] DELETE /identity/v3/projects/57243a254e864281a3d77c2b8d160f1b => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:29:33.817977 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e393700-82d8-45eb-9924-e8672f88146f None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:33.818344 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e393700-82d8-45eb-9924-e8672f88146f None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:33.818642 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e393700-82d8-45eb-9924-e8672f88146f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:33.841726 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-8e393700-82d8-45eb-9924-e8672f88146f None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:33.843952 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-8e393700-82d8-45eb-9924-e8672f88146f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:33.917366 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1125/2273] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:29:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:29:33.926673 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ff563-5350-4c38-a634-043722fc7811 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:33.927261 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ff563-5350-4c38-a634-043722fc7811 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:33.927531 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ff563-5350-4c38-a634-043722fc7811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:33.952656 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-9a0ff563-5350-4c38-a634-043722fc7811 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:33.954311 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-9a0ff563-5350-4c38-a634-043722fc7811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:33.963474 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1149/2274] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:29:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:29:33.968528 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e393700-82d8-45eb-9924-e8672f88146f None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:33.983165 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e393700-82d8-45eb-9924-e8672f88146f None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:33.998006 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34db92-ee01-4a48-aefb-00dc5b662ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:33.998961 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34db92-ee01-4a48-aefb-00dc5b662ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:33.999314 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34db92-ee01-4a48-aefb-00dc5b662ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:33.999596 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34db92-ee01-4a48-aefb-00dc5b662ca7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:34.000374 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec34db92-ee01-4a48-aefb-00dc5b662ca7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:29:34.000689 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec34db92-ee01-4a48-aefb-00dc5b662ca7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:34.003697 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec34db92-ee01-4a48-aefb-00dc5b662ca7 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:34.025877 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1126/2275] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:29:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:29:34.030253 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ff563-5350-4c38-a634-043722fc7811 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:34.040839 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ff563-5350-4c38-a634-043722fc7811 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:34.049936 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff1a640-05f3-48e5-bc8d-1dea0a974871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:34.050792 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff1a640-05f3-48e5-bc8d-1dea0a974871 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:34.051253 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff1a640-05f3-48e5-bc8d-1dea0a974871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:34.051699 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff1a640-05f3-48e5-bc8d-1dea0a974871 admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:34.052614 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff1a640-05f3-48e5-bc8d-1dea0a974871 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:34.055464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff1a640-05f3-48e5-bc8d-1dea0a974871 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:34.118946 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1150/2276] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:29:34 2022] POST /identity/v3/projects => generated 377 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:29:34.124067 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34db92-ee01-4a48-aefb-00dc5b662ca7 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:34.143391 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34db92-ee01-4a48-aefb-00dc5b662ca7 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:34.229969 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a46c1-7b62-41f3-be6b-307bfe198e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:34.230600 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a46c1-7b62-41f3-be6b-307bfe198e3b admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:34.230882 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a46c1-7b62-41f3-be6b-307bfe198e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:34.231158 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a46c1-7b62-41f3-be6b-307bfe198e3b admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:34.231957 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76a46c1-7b62-41f3-be6b-307bfe198e3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:34.234715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76a46c1-7b62-41f3-be6b-307bfe198e3b admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:34.261233 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1127/2277] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:29:34 2022] POST /identity/v3/users => generated 289 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:29:34.265302 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff1a640-05f3-48e5-bc8d-1dea0a974871 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:34.277618 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff1a640-05f3-48e5-bc8d-1dea0a974871 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:34.285730 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc9136-dd9b-47a9-aa3d-0b1873849ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:34.286769 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc9136-dd9b-47a9-aa3d-0b1873849ec2 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:34.287177 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc9136-dd9b-47a9-aa3d-0b1873849ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:34.287536 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc9136-dd9b-47a9-aa3d-0b1873849ec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:34.289049 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc9136-dd9b-47a9-aa3d-0b1873849ec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:34.290282 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc9136-dd9b-47a9-aa3d-0b1873849ec2 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:34.306319 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1151/2278] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:29:34.307618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a46c1-7b62-41f3-be6b-307bfe198e3b admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:34.322272 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a46c1-7b62-41f3-be6b-307bfe198e3b admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:34.327328 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014bfbf5-0de6-4f4b-83ed-896f201c8ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:34.328019 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014bfbf5-0de6-4f4b-83ed-896f201c8ccd admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:34.328303 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014bfbf5-0de6-4f4b-83ed-896f201c8ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:34.328579 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014bfbf5-0de6-4f4b-83ed-896f201c8ccd admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:34.329543 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014bfbf5-0de6-4f4b-83ed-896f201c8ccd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:34.331191 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014bfbf5-0de6-4f4b-83ed-896f201c8ccd admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:34.339733 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:34.346947 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1128/2279] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:29:34.351469 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc9136-dd9b-47a9-aa3d-0b1873849ec2 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:34.369750 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc9136-dd9b-47a9-aa3d-0b1873849ec2 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:34.376874 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85c0c0-d3b7-4507-ab30-8e3accce2d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:34.378789 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85c0c0-d3b7-4507-ab30-8e3accce2d67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:34.379377 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85c0c0-d3b7-4507-ab30-8e3accce2d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:34.380029 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e85c0c0-d3b7-4507-ab30-8e3accce2d67 admin admin] PATH_INFO: `/v3/projects/5323b9f1fdd049209dcd6f6c17abbf5f/users/3a61b98875234dba9b43d7ee8a95fcb4/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:34.404735 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:34.409810 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e85c0c0-d3b7-4507-ab30-8e3accce2d67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5323b9f1fdd049209dcd6f6c17abbf5f, user_id=3a61b98875234dba9b43d7ee8a95fcb4, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:34.413419 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e85c0c0-d3b7-4507-ab30-8e3accce2d67 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:34.432841 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1152/2280] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:29:34 2022] PUT /identity/v3/projects/5323b9f1fdd049209dcd6f6c17abbf5f/users/3a61b98875234dba9b43d7ee8a95fcb4/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:29:34.446785 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8da09-14f3-458e-be42-1dcae2c9cb14 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:34.447150 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8da09-14f3-458e-be42-1dcae2c9cb14 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:34.447511 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df8da09-14f3-458e-be42-1dcae2c9cb14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:34.485455 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-0df8da09-14f3-458e-be42-1dcae2c9cb14 None None] MFA Rules not processed for user `3a61b98875234dba9b43d7ee8a95fcb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:34.487283 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-0df8da09-14f3-458e-be42-1dcae2c9cb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:34.574412 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1129/2281] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:29:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:29:34.579406 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85c0c0-d3b7-4507-ab30-8e3accce2d67 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:34.671581 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e85c0c0-d3b7-4507-ab30-8e3accce2d67 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:34.677272 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985d9f1-3d6d-49f1-bdca-d683ae04faf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:34.678412 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985d9f1-3d6d-49f1-bdca-d683ae04faf9 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:34.678412 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985d9f1-3d6d-49f1-bdca-d683ae04faf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:34.679232 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985d9f1-3d6d-49f1-bdca-d683ae04faf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:34.680024 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1985d9f1-3d6d-49f1-bdca-d683ae04faf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:34.687376 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1985d9f1-3d6d-49f1-bdca-d683ae04faf9 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:34.737274 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1153/2282] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:29:34 2022] POST /identity/v3/projects => generated 377 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:29:34.738371 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df8da09-14f3-458e-be42-1dcae2c9cb14 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:34.753703 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df8da09-14f3-458e-be42-1dcae2c9cb14 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:34.841186 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e935f5bb-76a9-4bb6-9938-d2094defb069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:34.844293 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e935f5bb-76a9-4bb6-9938-d2094defb069 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:34.845060 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e935f5bb-76a9-4bb6-9938-d2094defb069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:34.845707 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e935f5bb-76a9-4bb6-9938-d2094defb069 admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:34.847315 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e935f5bb-76a9-4bb6-9938-d2094defb069 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:34.849719 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e935f5bb-76a9-4bb6-9938-d2094defb069 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:34.878722 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1130/2283] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:29:34 2022] POST /identity/v3/users => generated 295 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:29:34.882872 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985d9f1-3d6d-49f1-bdca-d683ae04faf9 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:34.895035 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985d9f1-3d6d-49f1-bdca-d683ae04faf9 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:34.899901 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30966d62-0ab2-4abd-97c7-73b122939e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:34.900597 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30966d62-0ab2-4abd-97c7-73b122939e93 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:34.900876 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30966d62-0ab2-4abd-97c7-73b122939e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:34.901187 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30966d62-0ab2-4abd-97c7-73b122939e93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:34.901917 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30966d62-0ab2-4abd-97c7-73b122939e93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:34.903536 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30966d62-0ab2-4abd-97c7-73b122939e93 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:34.919384 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1154/2284] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:29:34.924727 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e935f5bb-76a9-4bb6-9938-d2094defb069 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:34.935761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e935f5bb-76a9-4bb6-9938-d2094defb069 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:34.941536 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a9aa27-df75-44d2-9dc3-9616891c9ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:34.942603 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9aa27-df75-44d2-9dc3-9616891c9ca8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:34.942897 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9aa27-df75-44d2-9dc3-9616891c9ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:34.943156 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a9aa27-df75-44d2-9dc3-9616891c9ca8 admin admin] PATH_INFO: `/v3/projects/bb4ad11343fa4e8796a8439ee53fcb47/users/26bb078afe9a4e799c8be6dd805e72ed/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:34.967566 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a9aa27-df75-44d2-9dc3-9616891c9ca8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb4ad11343fa4e8796a8439ee53fcb47, user_id=26bb078afe9a4e799c8be6dd805e72ed, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:34.971432 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a9aa27-df75-44d2-9dc3-9616891c9ca8 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:34.989463 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1131/2285] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:29:34 2022] PUT /identity/v3/projects/bb4ad11343fa4e8796a8439ee53fcb47/users/26bb078afe9a4e799c8be6dd805e72ed/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:29:34.993615 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30966d62-0ab2-4abd-97c7-73b122939e93 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:35.009410 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30966d62-0ab2-4abd-97c7-73b122939e93 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:35.081217 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ad3841-acbb-41dd-adc0-80e2443593cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:35.082094 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad3841-acbb-41dd-adc0-80e2443593cb admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:35.082376 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad3841-acbb-41dd-adc0-80e2443593cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:35.082649 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad3841-acbb-41dd-adc0-80e2443593cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:35.083480 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ad3841-acbb-41dd-adc0-80e2443593cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:35.085250 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ad3841-acbb-41dd-adc0-80e2443593cb admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:35.100788 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1155/2286] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:29:35.105406 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a9aa27-df75-44d2-9dc3-9616891c9ca8 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:35.144101 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a9aa27-df75-44d2-9dc3-9616891c9ca8 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:35.151688 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220d968a-d69d-493d-805f-218dfcf70fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:35.153084 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d968a-d69d-493d-805f-218dfcf70fd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:35.153593 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d968a-d69d-493d-805f-218dfcf70fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:35.154107 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d968a-d69d-493d-805f-218dfcf70fd2 admin admin] PATH_INFO: `/v3/projects/bb4ad11343fa4e8796a8439ee53fcb47/users/26bb078afe9a4e799c8be6dd805e72ed/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:35.157566 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220d968a-d69d-493d-805f-218dfcf70fd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb4ad11343fa4e8796a8439ee53fcb47, user_id=26bb078afe9a4e799c8be6dd805e72ed, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:35.162261 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220d968a-d69d-493d-805f-218dfcf70fd2 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:35.175923 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1132/2287] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:29:35 2022] PUT /identity/v3/projects/bb4ad11343fa4e8796a8439ee53fcb47/users/26bb078afe9a4e799c8be6dd805e72ed/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:29:35.187065 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b1cba2-e796-4bca-a031-ff9ada6ea1a7 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:35.187438 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b1cba2-e796-4bca-a031-ff9ada6ea1a7 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:35.188374 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b1cba2-e796-4bca-a031-ff9ada6ea1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:35.246754 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-48b1cba2-e796-4bca-a031-ff9ada6ea1a7 None None] MFA Rules not processed for user `26bb078afe9a4e799c8be6dd805e72ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:35.248821 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-48b1cba2-e796-4bca-a031-ff9ada6ea1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:35.340655 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:35.373006 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1156/2288] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:29:35 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:29:35.386170 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220d968a-d69d-493d-805f-218dfcf70fd2 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:35.388843 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-220d968a-d69d-493d-805f-218dfcf70fd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:35.467031 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:35.489576 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220d968a-d69d-493d-805f-218dfcf70fd2 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:35.494610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c286b39-7be5-4959-ad0f-b0119d20f748 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:35.495468 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c286b39-7be5-4959-ad0f-b0119d20f748 service cinder] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:35.495745 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c286b39-7be5-4959-ad0f-b0119d20f748 service cinder] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:35.496016 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c286b39-7be5-4959-ad0f-b0119d20f748 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:35.565025 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c286b39-7be5-4959-ad0f-b0119d20f748 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:35.566548 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c286b39-7be5-4959-ad0f-b0119d20f748 service cinder] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:35.580073 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1133/2289] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:29:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:29:41.281372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9871cfe-de96-4823-8212-44422648591e None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:41.282298 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9871cfe-de96-4823-8212-44422648591e None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:41.282741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9871cfe-de96-4823-8212-44422648591e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:41.305639 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-f9871cfe-de96-4823-8212-44422648591e None None] MFA Rules not processed for user `86a18e313e4944748aef58d54bfe7b13`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:41.307645 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-f9871cfe-de96-4823-8212-44422648591e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:41.378354 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1157/2290] 10.222.0.50 () {58 vars in 1009 bytes} [Fri May 27 03:29:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:29:41.386548 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c286b39-7be5-4959-ad0f-b0119d20f748 service cinder] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:41.397347 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c286b39-7be5-4959-ad0f-b0119d20f748 service cinder] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:41.402272 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf20f0e-98e5-4ace-9fe7-0a8909f3d1e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:41.402886 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf20f0e-98e5-4ace-9fe7-0a8909f3d1e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:41.403230 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf20f0e-98e5-4ace-9fe7-0a8909f3d1e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:41.403572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf20f0e-98e5-4ace-9fe7-0a8909f3d1e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:41.405951 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9871cfe-de96-4823-8212-44422648591e None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:41.407607 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-f9871cfe-de96-4823-8212-44422648591e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:41.414470 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf20f0e-98e5-4ace-9fe7-0a8909f3d1e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:41.416192 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf20f0e-98e5-4ace-9fe7-0a8909f3d1e3 service cinder] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:41.427999 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1134/2291] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:29:41 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:29:41.486035 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9871cfe-de96-4823-8212-44422648591e None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:41.493092 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e8acc-28cf-465c-9353-f844c795a455 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de53e2f0fba4e0fac70887067c1963d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:41.493949 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e8acc-28cf-465c-9353-f844c795a455 service swift] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:41.494292 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e8acc-28cf-465c-9353-f844c795a455 service swift] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:41.494676 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e8acc-28cf-465c-9353-f844c795a455 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:41.506957 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144e8acc-28cf-465c-9353-f844c795a455 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:41.509114 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144e8acc-28cf-465c-9353-f844c795a455 service swift] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:41.526791 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1158/2292] 10.222.0.50 () {60 vars in 1384 bytes} [Fri May 27 03:29:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 27 03:29:51.423845 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=35 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:51.514521 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=63 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:29:58.356266 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162826e-e006-4748-b87b-46228fa733b6 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:58.357019 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162826e-e006-4748-b87b-46228fa733b6 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:58.358722 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162826e-e006-4748-b87b-46228fa733b6 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:58.361503 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1135/2293] 10.222.0.50 () {54 vars in 921 bytes} [Fri May 27 03:29:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:58.367348 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1043f29-e8e0-4c84-b9ac-054f746f38d8 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:58.367810 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1043f29-e8e0-4c84-b9ac-054f746f38d8 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:58.368123 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1043f29-e8e0-4c84-b9ac-054f746f38d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:58.388169 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-e1043f29-e8e0-4c84-b9ac-054f746f38d8 None None] MFA Rules not processed for user `1a699988f08a4001b3b7d764f01f4fc5`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:58.389523 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-e1043f29-e8e0-4c84-b9ac-054f746f38d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:58.475059 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1159/2294] 10.222.0.50 () {58 vars in 1018 bytes} [Fri May 27 03:29:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:29:58.488504 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f162826e-e006-4748-b87b-46228fa733b6 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:58.490397 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-f162826e-e006-4748-b87b-46228fa733b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:58.552333 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f162826e-e006-4748-b87b-46228fa733b6 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:58.562819 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267df8ce-8044-4ea2-bcab-4bb66c2d52be service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3c9c5c8d2e4376b1b70d974d4cb851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:58.563618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267df8ce-8044-4ea2-bcab-4bb66c2d52be service glance] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:58.563804 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267df8ce-8044-4ea2-bcab-4bb66c2d52be service glance] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:58.564086 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267df8ce-8044-4ea2-bcab-4bb66c2d52be service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:58.577050 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267df8ce-8044-4ea2-bcab-4bb66c2d52be service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:58.578842 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267df8ce-8044-4ea2-bcab-4bb66c2d52be service glance] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:58.595621 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1136/2295] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:29:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:29:58.687682 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069dd1b-0c99-4ea0-b193-6704c7c24b9d None None] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:58.688216 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069dd1b-0c99-4ea0-b193-6704c7c24b9d None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:58.688676 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9069dd1b-0c99-4ea0-b193-6704c7c24b9d None None] PATH_INFO: `/` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:58.690757 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1160/2296] 10.222.0.50 () {54 vars in 921 bytes} [Fri May 27 03:29:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:58.697624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682ba2a-2112-4fed-92d6-df1de3aeeeb4 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:58.697940 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682ba2a-2112-4fed-92d6-df1de3aeeeb4 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:58.698237 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682ba2a-2112-4fed-92d6-df1de3aeeeb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:58.722502 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-c682ba2a-2112-4fed-92d6-df1de3aeeeb4 None None] MFA Rules not processed for user `4ee30c0d6d694ee8945800157a9fc5b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:58.723704 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-c682ba2a-2112-4fed-92d6-df1de3aeeeb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:58.785549 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1137/2297] 10.222.0.50 () {58 vars in 1018 bytes} [Fri May 27 03:29:58 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:29:58.798535 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9069dd1b-0c99-4ea0-b193-6704c7c24b9d None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:58.802884 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-9069dd1b-0c99-4ea0-b193-6704c7c24b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:58.816661 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9069dd1b-0c99-4ea0-b193-6704c7c24b9d None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:58.820854 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b3654d-7b11-4cb2-ac72-e62f7d7729e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:58.821532 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b3654d-7b11-4cb2-ac72-e62f7d7729e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:58.821816 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b3654d-7b11-4cb2-ac72-e62f7d7729e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:58.822125 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b3654d-7b11-4cb2-ac72-e62f7d7729e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:58.836380 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b3654d-7b11-4cb2-ac72-e62f7d7729e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:58.838200 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b3654d-7b11-4cb2-ac72-e62f7d7729e0 service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:58.859264 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1161/2298] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:29:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:29:58.954570 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66eba8-88b5-4075-8789-a9eb2e6713f1 None None] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:58.954781 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66eba8-88b5-4075-8789-a9eb2e6713f1 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:58.955062 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66eba8-88b5-4075-8789-a9eb2e6713f1 None None] PATH_INFO: `/` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:58.956631 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1138/2299] 10.222.0.50 () {54 vars in 921 bytes} [Fri May 27 03:29:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 03:29:58.963588 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e0c6e-d03d-45f6-a37a-03584598998e None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:58.963874 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e0c6e-d03d-45f6-a37a-03584598998e None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:58.964143 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e0c6e-d03d-45f6-a37a-03584598998e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:58.983461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-d59e0c6e-d03d-45f6-a37a-03584598998e None None] MFA Rules not processed for user `1a699988f08a4001b3b7d764f01f4fc5`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:29:58.984662 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-d59e0c6e-d03d-45f6-a37a-03584598998e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:29:58.994191 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1162/2300] 10.222.0.50 () {58 vars in 1018 bytes} [Fri May 27 03:29:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:29:59.006718 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c66eba8-88b5-4075-8789-a9eb2e6713f1 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:29:59.018458 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c66eba8-88b5-4075-8789-a9eb2e6713f1 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:29:59.022728 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52e198-9a13-4aa2-99bf-07d980629627 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:29:59.023367 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52e198-9a13-4aa2-99bf-07d980629627 service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:29:59.023646 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52e198-9a13-4aa2-99bf-07d980629627 service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:29:59.023918 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52e198-9a13-4aa2-99bf-07d980629627 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:29:59.034425 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c52e198-9a13-4aa2-99bf-07d980629627 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:29:59.036420 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c52e198-9a13-4aa2-99bf-07d980629627 service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:29:59.047210 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1139/2301] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:29:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:30:05.717715 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81a7b2-7b17-47c5-9c3a-d8814d281de6 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:05.718198 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81a7b2-7b17-47c5-9c3a-d8814d281de6 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:05.718285 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81a7b2-7b17-47c5-9c3a-d8814d281de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:05.741408 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-0d81a7b2-7b17-47c5-9c3a-d8814d281de6 None None] MFA Rules not processed for user `86a18e313e4944748aef58d54bfe7b13`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:05.742894 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-0d81a7b2-7b17-47c5-9c3a-d8814d281de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:05.750570 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1163/2302] 10.222.0.50 () {58 vars in 1009 bytes} [Fri May 27 03:30:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:05.763426 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52e198-9a13-4aa2-99bf-07d980629627 service neutron] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:05.776385 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52e198-9a13-4aa2-99bf-07d980629627 service neutron] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:05.782194 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b3da49-4644-4cb0-997b-963f8c9df0bb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de53e2f0fba4e0fac70887067c1963d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:05.782971 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3da49-4644-4cb0-997b-963f8c9df0bb service swift] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:05.785834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3da49-4644-4cb0-997b-963f8c9df0bb service swift] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:05.786144 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3da49-4644-4cb0-997b-963f8c9df0bb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:05.799221 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b3da49-4644-4cb0-997b-963f8c9df0bb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:05.800840 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b3da49-4644-4cb0-997b-963f8c9df0bb service swift] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:05.809917 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1140/2303] 10.222.0.50 () {60 vars in 1384 bytes} [Fri May 27 03:30:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:30:06.872937 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81a7b2-7b17-47c5-9c3a-d8814d281de6 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:06.945814 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81a7b2-7b17-47c5-9c3a-d8814d281de6 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:06.950310 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede32c7-8f29-4a67-96b0-ab438977dbbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:06.951370 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede32c7-8f29-4a67-96b0-ab438977dbbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:06.951651 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede32c7-8f29-4a67-96b0-ab438977dbbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:06.951931 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aede32c7-8f29-4a67-96b0-ab438977dbbe admin admin] PATH_INFO: `/v3/users/3a61b98875234dba9b43d7ee8a95fcb4` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:06.953177 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede32c7-8f29-4a67-96b0-ab438977dbbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a61b98875234dba9b43d7ee8a95fcb4)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:06.955133 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aede32c7-8f29-4a67-96b0-ab438977dbbe admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:07.006649 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-aede32c7-8f29-4a67-96b0-ab438977dbbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a61b98875234dba9b43d7ee8a95fcb4'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:07.007401 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-aede32c7-8f29-4a67-96b0-ab438977dbbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a61b98875234dba9b43d7ee8a95fcb4'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:07.017208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-aede32c7-8f29-4a67-96b0-ab438977dbbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a61b98875234dba9b43d7ee8a95fcb4'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:07.018003 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-aede32c7-8f29-4a67-96b0-ab438977dbbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a61b98875234dba9b43d7ee8a95fcb4'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:07.030222 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:07.035528 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-aede32c7-8f29-4a67-96b0-ab438977dbbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a61b98875234dba9b43d7ee8a95fcb4'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:07.051220 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1164/2304] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:30:06 2022] DELETE /identity/v3/users/3a61b98875234dba9b43d7ee8a95fcb4 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:07.056447 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1f7eb-9ca3-4caf-be45-2011a513f7c6 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:07.056988 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1f7eb-9ca3-4caf-be45-2011a513f7c6 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:07.057088 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1f7eb-9ca3-4caf-be45-2011a513f7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:07.080682 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-7dc1f7eb-9ca3-4caf-be45-2011a513f7c6 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:07.081931 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-7dc1f7eb-9ca3-4caf-be45-2011a513f7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:07.177736 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1141/2305] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:07.186427 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede32c7-8f29-4a67-96b0-ab438977dbbe admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:07.194383 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-aede32c7-8f29-4a67-96b0-ab438977dbbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:07.348011 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aede32c7-8f29-4a67-96b0-ab438977dbbe admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:07.367435 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e60e0ef-35ce-47ab-8f27-562e9157f354 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:07.368134 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60e0ef-35ce-47ab-8f27-562e9157f354 service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:07.368418 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60e0ef-35ce-47ab-8f27-562e9157f354 service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:07.368690 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60e0ef-35ce-47ab-8f27-562e9157f354 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:07.383768 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e60e0ef-35ce-47ab-8f27-562e9157f354 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:07.385437 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e60e0ef-35ce-47ab-8f27-562e9157f354 service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:07.410243 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1165/2306] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:30:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:30:08.048792 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1f7eb-9ca3-4caf-be45-2011a513f7c6 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:08.050615 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-7dc1f7eb-9ca3-4caf-be45-2011a513f7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:08.063520 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc1f7eb-9ca3-4caf-be45-2011a513f7c6 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:08.070298 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86f137-9c97-4cc5-8a38-521bfa26f783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:08.071558 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86f137-9c97-4cc5-8a38-521bfa26f783 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:08.071890 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86f137-9c97-4cc5-8a38-521bfa26f783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:08.072169 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa86f137-9c97-4cc5-8a38-521bfa26f783 admin admin] PATH_INFO: `/v3/projects/5323b9f1fdd049209dcd6f6c17abbf5f` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:08.073456 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa86f137-9c97-4cc5-8a38-521bfa26f783 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5323b9f1fdd049209dcd6f6c17abbf5f)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:08.075552 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa86f137-9c97-4cc5-8a38-521bfa26f783 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:08.117836 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:08.154158 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-fa86f137-9c97-4cc5-8a38-521bfa26f783 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5323b9f1fdd049209dcd6f6c17abbf5f', 'request_id': 'req-fa86f137-9c97-4cc5-8a38-521bfa26f783'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:08.161610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-fa86f137-9c97-4cc5-8a38-521bfa26f783 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5323b9f1fdd049209dcd6f6c17abbf5f', 'request_id': 'req-fa86f137-9c97-4cc5-8a38-521bfa26f783'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:08.167589 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-fa86f137-9c97-4cc5-8a38-521bfa26f783 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5323b9f1fdd049209dcd6f6c17abbf5f', 'request_id': 'req-fa86f137-9c97-4cc5-8a38-521bfa26f783'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:08.183465 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-fa86f137-9c97-4cc5-8a38-521bfa26f783 admin admin] The token cache is being invalidate because project 5323b9f1fdd049209dcd6f6c17abbf5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:30:08.183749 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-fa86f137-9c97-4cc5-8a38-521bfa26f783 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:08.184386 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-fa86f137-9c97-4cc5-8a38-521bfa26f783 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:08.185983 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1142/2307] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:30:08 2022] DELETE /identity/v3/projects/5323b9f1fdd049209dcd6f6c17abbf5f => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:08.190211 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e60e0ef-35ce-47ab-8f27-562e9157f354 service neutron] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:08.200233 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e60e0ef-35ce-47ab-8f27-562e9157f354 service neutron] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:08.266927 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3451a73e-1d73-4e6f-9a57-bdc6cb3773bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:08.267654 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451a73e-1d73-4e6f-9a57-bdc6cb3773bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:08.267966 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451a73e-1d73-4e6f-9a57-bdc6cb3773bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:08.268230 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3451a73e-1d73-4e6f-9a57-bdc6cb3773bc admin admin] PATH_INFO: `/v3/users/26bb078afe9a4e799c8be6dd805e72ed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:08.270120 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3451a73e-1d73-4e6f-9a57-bdc6cb3773bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=26bb078afe9a4e799c8be6dd805e72ed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:08.272464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3451a73e-1d73-4e6f-9a57-bdc6cb3773bc admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:08.345414 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3451a73e-1d73-4e6f-9a57-bdc6cb3773bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26bb078afe9a4e799c8be6dd805e72ed'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:08.349878 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3451a73e-1d73-4e6f-9a57-bdc6cb3773bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26bb078afe9a4e799c8be6dd805e72ed'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:08.367562 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3451a73e-1d73-4e6f-9a57-bdc6cb3773bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26bb078afe9a4e799c8be6dd805e72ed'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:08.368203 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3451a73e-1d73-4e6f-9a57-bdc6cb3773bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26bb078afe9a4e799c8be6dd805e72ed'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:08.383778 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3451a73e-1d73-4e6f-9a57-bdc6cb3773bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26bb078afe9a4e799c8be6dd805e72ed'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:08.396388 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1166/2308] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:30:08 2022] DELETE /identity/v3/users/26bb078afe9a4e799c8be6dd805e72ed => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:08.787465 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86f137-9c97-4cc5-8a38-521bfa26f783 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:08.789726 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-fa86f137-9c97-4cc5-8a38-521bfa26f783 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:08.809975 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa86f137-9c97-4cc5-8a38-521bfa26f783 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:08.877834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd9266-4a1e-47f6-b035-7bd647265eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:08.878721 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd9266-4a1e-47f6-b035-7bd647265eda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:08.879000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd9266-4a1e-47f6-b035-7bd647265eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:08.879278 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd9266-4a1e-47f6-b035-7bd647265eda admin admin] PATH_INFO: `/v3/projects/bb4ad11343fa4e8796a8439ee53fcb47` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:08.880471 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd9266-4a1e-47f6-b035-7bd647265eda admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb4ad11343fa4e8796a8439ee53fcb47)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:08.883344 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd9266-4a1e-47f6-b035-7bd647265eda admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:08.937761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-f7dd9266-4a1e-47f6-b035-7bd647265eda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb4ad11343fa4e8796a8439ee53fcb47', 'request_id': 'req-f7dd9266-4a1e-47f6-b035-7bd647265eda'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:08.945295 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-f7dd9266-4a1e-47f6-b035-7bd647265eda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb4ad11343fa4e8796a8439ee53fcb47', 'request_id': 'req-f7dd9266-4a1e-47f6-b035-7bd647265eda'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:08.953464 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-f7dd9266-4a1e-47f6-b035-7bd647265eda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb4ad11343fa4e8796a8439ee53fcb47', 'request_id': 'req-f7dd9266-4a1e-47f6-b035-7bd647265eda'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:08.970864 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-f7dd9266-4a1e-47f6-b035-7bd647265eda admin admin] The token cache is being invalidate because project bb4ad11343fa4e8796a8439ee53fcb47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:30:08.971179 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-f7dd9266-4a1e-47f6-b035-7bd647265eda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:08.971834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-f7dd9266-4a1e-47f6-b035-7bd647265eda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:08.973445 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1143/2309] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:30:08 2022] DELETE /identity/v3/projects/bb4ad11343fa4e8796a8439ee53fcb47 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:08.979082 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d5440-d6e5-47d4-bcdd-694534d37bb1 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:08.979489 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d5440-d6e5-47d4-bcdd-694534d37bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:08.980179 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d5440-d6e5-47d4-bcdd-694534d37bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:08.994327 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:09.004448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-188d5440-d6e5-47d4-bcdd-694534d37bb1 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:09.007177 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-188d5440-d6e5-47d4-bcdd-694534d37bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:09.079635 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1167/2310] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:09.087402 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750a200-44b8-496d-9350-0097f2586e58 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:09.087987 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750a200-44b8-496d-9350-0097f2586e58 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:09.087987 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750a200-44b8-496d-9350-0097f2586e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:09.107364 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-3750a200-44b8-496d-9350-0097f2586e58 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:09.108710 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-3750a200-44b8-496d-9350-0097f2586e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:09.120985 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1144/2311] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:09.124056 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188d5440-d6e5-47d4-bcdd-694534d37bb1 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:09.135478 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188d5440-d6e5-47d4-bcdd-694534d37bb1 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:09.141560 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f955774-6ee8-4a1c-bd52-653f230cd048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:09.142579 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f955774-6ee8-4a1c-bd52-653f230cd048 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:09.143096 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f955774-6ee8-4a1c-bd52-653f230cd048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:09.143581 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f955774-6ee8-4a1c-bd52-653f230cd048 admin admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:09.144554 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f955774-6ee8-4a1c-bd52-653f230cd048 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:09.145110 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f955774-6ee8-4a1c-bd52-653f230cd048 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:09.147136 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f955774-6ee8-4a1c-bd52-653f230cd048 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:09.161675 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1168/2312] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:30:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:09.166810 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3750a200-44b8-496d-9350-0097f2586e58 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:09.177566 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3750a200-44b8-496d-9350-0097f2586e58 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:09.182943 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445eecf4-9367-46d1-8015-2092fbe622b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:09.183625 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445eecf4-9367-46d1-8015-2092fbe622b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:09.183897 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445eecf4-9367-46d1-8015-2092fbe622b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:09.184179 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445eecf4-9367-46d1-8015-2092fbe622b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:09.184882 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445eecf4-9367-46d1-8015-2092fbe622b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:09.186881 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445eecf4-9367-46d1-8015-2092fbe622b6 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:09.222274 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1145/2313] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:30:09 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:09.231589 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f955774-6ee8-4a1c-bd52-653f230cd048 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:09.243347 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f955774-6ee8-4a1c-bd52-653f230cd048 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:09.321153 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2595da-7e99-413c-bd42-634bb009c09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:09.321874 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2595da-7e99-413c-bd42-634bb009c09e admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:09.322127 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2595da-7e99-413c-bd42-634bb009c09e admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:09.322420 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2595da-7e99-413c-bd42-634bb009c09e admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:09.323141 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2595da-7e99-413c-bd42-634bb009c09e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:09.325171 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2595da-7e99-413c-bd42-634bb009c09e admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:09.360730 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1169/2314] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:30:09 2022] POST /identity/v3/users => generated 289 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:09.362790 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445eecf4-9367-46d1-8015-2092fbe622b6 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:09.374192 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445eecf4-9367-46d1-8015-2092fbe622b6 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:09.378975 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83200e91-a738-43d9-9172-b6c21fe42790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:09.379650 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83200e91-a738-43d9-9172-b6c21fe42790 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:09.379931 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83200e91-a738-43d9-9172-b6c21fe42790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:09.380277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83200e91-a738-43d9-9172-b6c21fe42790 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:09.381040 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83200e91-a738-43d9-9172-b6c21fe42790 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:09.383761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83200e91-a738-43d9-9172-b6c21fe42790 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:09.392477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:09.404991 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1146/2315] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:09.408904 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2595da-7e99-413c-bd42-634bb009c09e admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:09.420099 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2595da-7e99-413c-bd42-634bb009c09e admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:09.426485 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec884028-498e-4efd-8754-14c875449c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:09.427189 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec884028-498e-4efd-8754-14c875449c9c admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:09.427468 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec884028-498e-4efd-8754-14c875449c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:09.427749 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec884028-498e-4efd-8754-14c875449c9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:09.428929 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec884028-498e-4efd-8754-14c875449c9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:09.430743 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec884028-498e-4efd-8754-14c875449c9c admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:09.441958 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1170/2316] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:30:09.446395 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83200e91-a738-43d9-9172-b6c21fe42790 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:09.457756 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83200e91-a738-43d9-9172-b6c21fe42790 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:09.462686 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866fab4d-6cc9-4d14-ad41-17680be19f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:09.463593 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fab4d-6cc9-4d14-ad41-17680be19f71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:09.463930 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fab4d-6cc9-4d14-ad41-17680be19f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:09.464364 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fab4d-6cc9-4d14-ad41-17680be19f71 admin admin] PATH_INFO: `/v3/projects/d5c45cdb8f1f45acabfa6999be0a6cc0/users/d960a7d226a343c58830d6979e76e2c2/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:09.481211 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866fab4d-6cc9-4d14-ad41-17680be19f71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5c45cdb8f1f45acabfa6999be0a6cc0, user_id=d960a7d226a343c58830d6979e76e2c2, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:09.483981 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866fab4d-6cc9-4d14-ad41-17680be19f71 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:09.498928 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1147/2317] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:30:09 2022] PUT /identity/v3/projects/d5c45cdb8f1f45acabfa6999be0a6cc0/users/d960a7d226a343c58830d6979e76e2c2/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:09.510597 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a109d-ef43-40f9-9599-2c04f1f0a2ce None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:09.510885 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a109d-ef43-40f9-9599-2c04f1f0a2ce None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:09.511158 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a109d-ef43-40f9-9599-2c04f1f0a2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:09.540349 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-436a109d-ef43-40f9-9599-2c04f1f0a2ce None None] MFA Rules not processed for user `d960a7d226a343c58830d6979e76e2c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:09.541599 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-436a109d-ef43-40f9-9599-2c04f1f0a2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:09.609608 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1171/2318] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:09.614591 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866fab4d-6cc9-4d14-ad41-17680be19f71 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:09.682002 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866fab4d-6cc9-4d14-ad41-17680be19f71 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:09.686835 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f287c4-efe4-469a-91ba-e8aa7a1101f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:09.687512 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f287c4-efe4-469a-91ba-e8aa7a1101f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:09.687793 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f287c4-efe4-469a-91ba-e8aa7a1101f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:09.688064 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f287c4-efe4-469a-91ba-e8aa7a1101f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:09.688759 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f287c4-efe4-469a-91ba-e8aa7a1101f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:09.690740 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f287c4-efe4-469a-91ba-e8aa7a1101f8 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:09.720775 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1148/2319] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:30:09 2022] POST /identity/v3/projects => generated 379 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:09.724716 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a109d-ef43-40f9-9599-2c04f1f0a2ce None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:09.730623 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:09.737478 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a109d-ef43-40f9-9599-2c04f1f0a2ce None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:09.839031 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e559ca7-a057-44de-9702-8fd24535de00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:09.839904 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e559ca7-a057-44de-9702-8fd24535de00 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:09.840183 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e559ca7-a057-44de-9702-8fd24535de00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:09.840462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e559ca7-a057-44de-9702-8fd24535de00 admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:09.841230 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e559ca7-a057-44de-9702-8fd24535de00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:09.843227 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e559ca7-a057-44de-9702-8fd24535de00 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:09.877458 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1172/2320] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:30:09 2022] POST /identity/v3/users => generated 296 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:30:09.879673 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f287c4-efe4-469a-91ba-e8aa7a1101f8 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:09.893087 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f287c4-efe4-469a-91ba-e8aa7a1101f8 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:09.901617 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd066412-70a6-4723-827a-ae76d8fda5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:09.902869 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd066412-70a6-4723-827a-ae76d8fda5ff admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:09.903202 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd066412-70a6-4723-827a-ae76d8fda5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:09.903486 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd066412-70a6-4723-827a-ae76d8fda5ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:09.904232 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd066412-70a6-4723-827a-ae76d8fda5ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:09.906530 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd066412-70a6-4723-827a-ae76d8fda5ff admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:09.920989 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1149/2321] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:09.924883 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e559ca7-a057-44de-9702-8fd24535de00 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:09.939721 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e559ca7-a057-44de-9702-8fd24535de00 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:09.944602 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d073e-8ab3-4080-bfb0-580f7608e4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:09.945521 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d073e-8ab3-4080-bfb0-580f7608e4ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:09.945798 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d073e-8ab3-4080-bfb0-580f7608e4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:09.946071 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d073e-8ab3-4080-bfb0-580f7608e4ef admin admin] PATH_INFO: `/v3/projects/8a48b70e4365461c96e8da8e0537ada6/users/8b6f8c2f2e4d4d61b2b8c60701e0788d/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:09.965604 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d073e-8ab3-4080-bfb0-580f7608e4ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a48b70e4365461c96e8da8e0537ada6, user_id=8b6f8c2f2e4d4d61b2b8c60701e0788d, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:09.968413 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d073e-8ab3-4080-bfb0-580f7608e4ef admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:09.984972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1173/2322] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:30:09 2022] PUT /identity/v3/projects/8a48b70e4365461c96e8da8e0537ada6/users/8b6f8c2f2e4d4d61b2b8c60701e0788d/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:09.989237 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd066412-70a6-4723-827a-ae76d8fda5ff admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:09.999726 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd066412-70a6-4723-827a-ae76d8fda5ff admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:10.063211 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e938d810-025f-4f79-a1ea-e1dc84db1e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:10.063904 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938d810-025f-4f79-a1ea-e1dc84db1e94 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:10.064218 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938d810-025f-4f79-a1ea-e1dc84db1e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:10.064462 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938d810-025f-4f79-a1ea-e1dc84db1e94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:10.065232 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e938d810-025f-4f79-a1ea-e1dc84db1e94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:10.067031 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e938d810-025f-4f79-a1ea-e1dc84db1e94 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:10.078590 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1150/2323] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:10.082621 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d073e-8ab3-4080-bfb0-580f7608e4ef admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:10.093303 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d073e-8ab3-4080-bfb0-580f7608e4ef admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:10.098130 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6809c1-c069-4caa-bce8-76e3703ecf85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:10.099106 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6809c1-c069-4caa-bce8-76e3703ecf85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:10.099458 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6809c1-c069-4caa-bce8-76e3703ecf85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:10.099882 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6809c1-c069-4caa-bce8-76e3703ecf85 admin admin] PATH_INFO: `/v3/projects/8a48b70e4365461c96e8da8e0537ada6/users/8b6f8c2f2e4d4d61b2b8c60701e0788d/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:10.102497 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6809c1-c069-4caa-bce8-76e3703ecf85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a48b70e4365461c96e8da8e0537ada6, user_id=8b6f8c2f2e4d4d61b2b8c60701e0788d, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:10.105248 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6809c1-c069-4caa-bce8-76e3703ecf85 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:10.117285 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1174/2324] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:30:10 2022] PUT /identity/v3/projects/8a48b70e4365461c96e8da8e0537ada6/users/8b6f8c2f2e4d4d61b2b8c60701e0788d/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:30:10.128195 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0271b-b138-4513-9bdb-8b9d80b60cba None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:10.128797 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0271b-b138-4513-9bdb-8b9d80b60cba None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:10.128897 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0271b-b138-4513-9bdb-8b9d80b60cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:10.167662 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-5fd0271b-b138-4513-9bdb-8b9d80b60cba None None] MFA Rules not processed for user `8b6f8c2f2e4d4d61b2b8c60701e0788d`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:10.168901 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-5fd0271b-b138-4513-9bdb-8b9d80b60cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:10.209531 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:10.265120 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1151/2325] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:10 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:10.278078 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6809c1-c069-4caa-bce8-76e3703ecf85 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:10.280375 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-7b6809c1-c069-4caa-bce8-76e3703ecf85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:10.402312 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6809c1-c069-4caa-bce8-76e3703ecf85 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:10.406834 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91457a13-f46f-4591-be5b-3b3d78772310 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:10.407822 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91457a13-f46f-4591-be5b-3b3d78772310 service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:10.408122 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91457a13-f46f-4591-be5b-3b3d78772310 service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:10.408464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91457a13-f46f-4591-be5b-3b3d78772310 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:10.423372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91457a13-f46f-4591-be5b-3b3d78772310 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:10.425046 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91457a13-f46f-4591-be5b-3b3d78772310 service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:10.437324 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1175/2326] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:30:10 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:30:10.824208 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0271b-b138-4513-9bdb-8b9d80b60cba None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:10.833613 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0271b-b138-4513-9bdb-8b9d80b60cba None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:10.838652 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91babb42-61ef-4dff-b27d-25b3ca18cfce tempest-VolumeQuotaClassesTest-311280616 tempest-VolumeQuotaClassesTest-311280616-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6f8c2f2e4d4d61b2b8c60701e0788d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a48b70e4365461c96e8da8e0537ada6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:10.839322 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91babb42-61ef-4dff-b27d-25b3ca18cfce tempest-VolumeQuotaClassesTest-311280616 tempest-VolumeQuotaClassesTest-311280616-project-admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:10.839604 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91babb42-61ef-4dff-b27d-25b3ca18cfce tempest-VolumeQuotaClassesTest-311280616 tempest-VolumeQuotaClassesTest-311280616-project-admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:10.839885 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91babb42-61ef-4dff-b27d-25b3ca18cfce tempest-VolumeQuotaClassesTest-311280616 tempest-VolumeQuotaClassesTest-311280616-project-admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:10.840629 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91babb42-61ef-4dff-b27d-25b3ca18cfce tempest-VolumeQuotaClassesTest-311280616 tempest-VolumeQuotaClassesTest-311280616-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:10.840972 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91babb42-61ef-4dff-b27d-25b3ca18cfce tempest-VolumeQuotaClassesTest-311280616 tempest-VolumeQuotaClassesTest-311280616-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:10.842601 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91babb42-61ef-4dff-b27d-25b3ca18cfce tempest-VolumeQuotaClassesTest-311280616 tempest-VolumeQuotaClassesTest-311280616-project-admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:10.856083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1152/2327] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:30:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:10.860474 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91457a13-f46f-4591-be5b-3b3d78772310 service cinder] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:10.870222 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91457a13-f46f-4591-be5b-3b3d78772310 service cinder] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:10.874849 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6802c55e-8c2b-4e2f-856e-282e08554ac4 tempest-VolumeQuotaClassesTest-311280616 tempest-VolumeQuotaClassesTest-311280616-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6f8c2f2e4d4d61b2b8c60701e0788d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a48b70e4365461c96e8da8e0537ada6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:10.875640 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802c55e-8c2b-4e2f-856e-282e08554ac4 tempest-VolumeQuotaClassesTest-311280616 tempest-VolumeQuotaClassesTest-311280616-project-admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:10.875925 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802c55e-8c2b-4e2f-856e-282e08554ac4 tempest-VolumeQuotaClassesTest-311280616 tempest-VolumeQuotaClassesTest-311280616-project-admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:10.876203 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802c55e-8c2b-4e2f-856e-282e08554ac4 tempest-VolumeQuotaClassesTest-311280616 tempest-VolumeQuotaClassesTest-311280616-project-admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:10.877070 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6802c55e-8c2b-4e2f-856e-282e08554ac4 tempest-VolumeQuotaClassesTest-311280616 tempest-VolumeQuotaClassesTest-311280616-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:10.879442 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6802c55e-8c2b-4e2f-856e-282e08554ac4 tempest-VolumeQuotaClassesTest-311280616 tempest-VolumeQuotaClassesTest-311280616-project-admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:10.914604 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1176/2328] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:30:10 2022] POST /identity/v3/projects => generated 355 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:10.918929 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91babb42-61ef-4dff-b27d-25b3ca18cfce tempest-VolumeQuotaClassesTest-311280616 tempest-VolumeQuotaClassesTest-311280616-project-admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:10.929832 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91babb42-61ef-4dff-b27d-25b3ca18cfce tempest-VolumeQuotaClassesTest-311280616 tempest-VolumeQuotaClassesTest-311280616-project-admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:10.999462 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e8994-fa03-4cc1-b735-a8d5f32fc98e tempest-VolumeQuotaClassesTest-311280616 tempest-VolumeQuotaClassesTest-311280616-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6f8c2f2e4d4d61b2b8c60701e0788d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a48b70e4365461c96e8da8e0537ada6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:11.000144 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e8994-fa03-4cc1-b735-a8d5f32fc98e tempest-VolumeQuotaClassesTest-311280616 tempest-VolumeQuotaClassesTest-311280616-project-admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:11.000493 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e8994-fa03-4cc1-b735-a8d5f32fc98e tempest-VolumeQuotaClassesTest-311280616 tempest-VolumeQuotaClassesTest-311280616-project-admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:11.000834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e8994-fa03-4cc1-b735-a8d5f32fc98e tempest-VolumeQuotaClassesTest-311280616 tempest-VolumeQuotaClassesTest-311280616-project-admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:11.001812 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e8994-fa03-4cc1-b735-a8d5f32fc98e tempest-VolumeQuotaClassesTest-311280616 tempest-VolumeQuotaClassesTest-311280616-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:11.002217 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e8994-fa03-4cc1-b735-a8d5f32fc98e tempest-VolumeQuotaClassesTest-311280616 tempest-VolumeQuotaClassesTest-311280616-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:11.004081 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e8994-fa03-4cc1-b735-a8d5f32fc98e tempest-VolumeQuotaClassesTest-311280616 tempest-VolumeQuotaClassesTest-311280616-project-admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:11.037258 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1153/2329] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:30:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:11.087235 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6802c55e-8c2b-4e2f-856e-282e08554ac4 tempest-VolumeQuotaClassesTest-311280616 tempest-VolumeQuotaClassesTest-311280616-project-admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:11.111101 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6802c55e-8c2b-4e2f-856e-282e08554ac4 tempest-VolumeQuotaClassesTest-311280616 tempest-VolumeQuotaClassesTest-311280616-project-admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:11.122724 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd42727-26bd-4bf9-842b-06c21709d855 tempest-VolumeQuotaClassesTest-311280616 tempest-VolumeQuotaClassesTest-311280616-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8b6f8c2f2e4d4d61b2b8c60701e0788d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8a48b70e4365461c96e8da8e0537ada6', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:11.123790 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd42727-26bd-4bf9-842b-06c21709d855 tempest-VolumeQuotaClassesTest-311280616 tempest-VolumeQuotaClassesTest-311280616-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:11.124168 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd42727-26bd-4bf9-842b-06c21709d855 tempest-VolumeQuotaClassesTest-311280616 tempest-VolumeQuotaClassesTest-311280616-project-admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:11.124546 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd42727-26bd-4bf9-842b-06c21709d855 tempest-VolumeQuotaClassesTest-311280616 tempest-VolumeQuotaClassesTest-311280616-project-admin] PATH_INFO: `/v3/projects/11a04b15cc374a6da57776a3005da3b3` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:11.126194 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd42727-26bd-4bf9-842b-06c21709d855 tempest-VolumeQuotaClassesTest-311280616 tempest-VolumeQuotaClassesTest-311280616-project-admin] RBAC: Authorizing `identity:delete_project(project_id=11a04b15cc374a6da57776a3005da3b3)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:11.128361 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd42727-26bd-4bf9-842b-06c21709d855 tempest-VolumeQuotaClassesTest-311280616 tempest-VolumeQuotaClassesTest-311280616-project-admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:11.190137 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=92,keystone:INSERT=6,keystone:DELETE=2 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:11.213616 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-1dd42727-26bd-4bf9-842b-06c21709d855 tempest-VolumeQuotaClassesTest-311280616 tempest-VolumeQuotaClassesTest-311280616-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11a04b15cc374a6da57776a3005da3b3', 'request_id': 'req-1dd42727-26bd-4bf9-842b-06c21709d855'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:11.220103 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-1dd42727-26bd-4bf9-842b-06c21709d855 tempest-VolumeQuotaClassesTest-311280616 tempest-VolumeQuotaClassesTest-311280616-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11a04b15cc374a6da57776a3005da3b3', 'request_id': 'req-1dd42727-26bd-4bf9-842b-06c21709d855'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:11.229951 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-1dd42727-26bd-4bf9-842b-06c21709d855 tempest-VolumeQuotaClassesTest-311280616 tempest-VolumeQuotaClassesTest-311280616-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11a04b15cc374a6da57776a3005da3b3', 'request_id': 'req-1dd42727-26bd-4bf9-842b-06c21709d855'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:11.242764 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-1dd42727-26bd-4bf9-842b-06c21709d855 tempest-VolumeQuotaClassesTest-311280616 tempest-VolumeQuotaClassesTest-311280616-project-admin] The token cache is being invalidate because project 11a04b15cc374a6da57776a3005da3b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:30:11.243099 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-1dd42727-26bd-4bf9-842b-06c21709d855 tempest-VolumeQuotaClassesTest-311280616 tempest-VolumeQuotaClassesTest-311280616-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:11.243776 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-1dd42727-26bd-4bf9-842b-06c21709d855 tempest-VolumeQuotaClassesTest-311280616 tempest-VolumeQuotaClassesTest-311280616-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:11.245402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1177/2330] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:30:11 2022] DELETE /identity/v3/projects/11a04b15cc374a6da57776a3005da3b3 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:11.475599 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e8994-fa03-4cc1-b735-a8d5f32fc98e tempest-VolumeQuotaClassesTest-311280616 tempest-VolumeQuotaClassesTest-311280616-project-admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:11.477474 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-041e8994-fa03-4cc1-b735-a8d5f32fc98e tempest-VolumeQuotaClassesTest-311280616 tempest-VolumeQuotaClassesTest-311280616-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:11.546323 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e8994-fa03-4cc1-b735-a8d5f32fc98e tempest-VolumeQuotaClassesTest-311280616 tempest-VolumeQuotaClassesTest-311280616-project-admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:11.618885 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a682c40-899e-4c66-ad4b-d2a4357fc11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:11.620074 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a682c40-899e-4c66-ad4b-d2a4357fc11d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:11.620541 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a682c40-899e-4c66-ad4b-d2a4357fc11d admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:11.621006 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a682c40-899e-4c66-ad4b-d2a4357fc11d admin admin] PATH_INFO: `/v3/users/d960a7d226a343c58830d6979e76e2c2` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:11.625850 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a682c40-899e-4c66-ad4b-d2a4357fc11d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d960a7d226a343c58830d6979e76e2c2)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:11.627931 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a682c40-899e-4c66-ad4b-d2a4357fc11d admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:11.629938 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:11.707674 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-9a682c40-899e-4c66-ad4b-d2a4357fc11d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd960a7d226a343c58830d6979e76e2c2'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:11.738692 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-9a682c40-899e-4c66-ad4b-d2a4357fc11d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd960a7d226a343c58830d6979e76e2c2'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:11.749779 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-9a682c40-899e-4c66-ad4b-d2a4357fc11d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd960a7d226a343c58830d6979e76e2c2'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:11.758545 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-9a682c40-899e-4c66-ad4b-d2a4357fc11d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd960a7d226a343c58830d6979e76e2c2'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:11.759472 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-9a682c40-899e-4c66-ad4b-d2a4357fc11d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd960a7d226a343c58830d6979e76e2c2'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:11.776846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1154/2331] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:30:11 2022] DELETE /identity/v3/users/d960a7d226a343c58830d6979e76e2c2 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:11.781507 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b7b52-fa77-4e94-9407-bfccc6302773 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:11.782033 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b7b52-fa77-4e94-9407-bfccc6302773 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:11.782033 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1b7b52-fa77-4e94-9407-bfccc6302773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:11.803216 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-fe1b7b52-fa77-4e94-9407-bfccc6302773 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:11.804371 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-fe1b7b52-fa77-4e94-9407-bfccc6302773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:11.872039 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1178/2332] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:11.882293 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a682c40-899e-4c66-ad4b-d2a4357fc11d admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:11.883819 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-9a682c40-899e-4c66-ad4b-d2a4357fc11d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:11.944511 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a682c40-899e-4c66-ad4b-d2a4357fc11d admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:11.948772 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080ea94-d3c7-4901-b1fa-34d45b5da75e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:11.949465 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080ea94-d3c7-4901-b1fa-34d45b5da75e service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:11.949744 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080ea94-d3c7-4901-b1fa-34d45b5da75e service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:11.950016 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080ea94-d3c7-4901-b1fa-34d45b5da75e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:11.960655 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f080ea94-d3c7-4901-b1fa-34d45b5da75e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:11.962181 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f080ea94-d3c7-4901-b1fa-34d45b5da75e service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:11.975499 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1155/2333] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:30:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:30:12.591609 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1b7b52-fa77-4e94-9407-bfccc6302773 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:12.593256 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-fe1b7b52-fa77-4e94-9407-bfccc6302773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:12.608225 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1b7b52-fa77-4e94-9407-bfccc6302773 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:12.618134 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c9041-45f9-4756-9c7f-8647e8039eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:12.619143 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c9041-45f9-4756-9c7f-8647e8039eee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:12.619299 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c9041-45f9-4756-9c7f-8647e8039eee admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:12.619570 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c9041-45f9-4756-9c7f-8647e8039eee admin admin] PATH_INFO: `/v3/projects/d5c45cdb8f1f45acabfa6999be0a6cc0` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:12.620796 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27c9041-45f9-4756-9c7f-8647e8039eee admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5c45cdb8f1f45acabfa6999be0a6cc0)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:12.622978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27c9041-45f9-4756-9c7f-8647e8039eee admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:12.680400 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f27c9041-45f9-4756-9c7f-8647e8039eee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5c45cdb8f1f45acabfa6999be0a6cc0', 'request_id': 'req-f27c9041-45f9-4756-9c7f-8647e8039eee'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:12.687141 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f27c9041-45f9-4756-9c7f-8647e8039eee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5c45cdb8f1f45acabfa6999be0a6cc0', 'request_id': 'req-f27c9041-45f9-4756-9c7f-8647e8039eee'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:12.698271 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f27c9041-45f9-4756-9c7f-8647e8039eee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5c45cdb8f1f45acabfa6999be0a6cc0', 'request_id': 'req-f27c9041-45f9-4756-9c7f-8647e8039eee'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:12.709591 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f27c9041-45f9-4756-9c7f-8647e8039eee admin admin] The token cache is being invalidate because project d5c45cdb8f1f45acabfa6999be0a6cc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:30:12.709872 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f27c9041-45f9-4756-9c7f-8647e8039eee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:12.710534 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f27c9041-45f9-4756-9c7f-8647e8039eee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:12.712126 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1179/2334] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:30:12 2022] DELETE /identity/v3/projects/d5c45cdb8f1f45acabfa6999be0a6cc0 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:12.716226 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080ea94-d3c7-4901-b1fa-34d45b5da75e service neutron] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:12.729148 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080ea94-d3c7-4901-b1fa-34d45b5da75e service neutron] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:12.794089 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae6b338-25e9-48b4-a003-7deaafcdb36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:12.795133 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6b338-25e9-48b4-a003-7deaafcdb36f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:12.795510 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6b338-25e9-48b4-a003-7deaafcdb36f admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:12.795946 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6b338-25e9-48b4-a003-7deaafcdb36f admin admin] PATH_INFO: `/v3/users/8b6f8c2f2e4d4d61b2b8c60701e0788d` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:12.797337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae6b338-25e9-48b4-a003-7deaafcdb36f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b6f8c2f2e4d4d61b2b8c60701e0788d)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:12.799646 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae6b338-25e9-48b4-a003-7deaafcdb36f admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:12.829281 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:12.865057 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-7ae6b338-25e9-48b4-a003-7deaafcdb36f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b6f8c2f2e4d4d61b2b8c60701e0788d'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:12.880924 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-7ae6b338-25e9-48b4-a003-7deaafcdb36f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b6f8c2f2e4d4d61b2b8c60701e0788d'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:12.898339 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-7ae6b338-25e9-48b4-a003-7deaafcdb36f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b6f8c2f2e4d4d61b2b8c60701e0788d'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:12.910564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-7ae6b338-25e9-48b4-a003-7deaafcdb36f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b6f8c2f2e4d4d61b2b8c60701e0788d'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:12.913866 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-7ae6b338-25e9-48b4-a003-7deaafcdb36f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b6f8c2f2e4d4d61b2b8c60701e0788d'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:12.946268 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1156/2335] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:30:12 2022] DELETE /identity/v3/users/8b6f8c2f2e4d4d61b2b8c60701e0788d => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:13.360217 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c9041-45f9-4756-9c7f-8647e8039eee admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:13.361661 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-f27c9041-45f9-4756-9c7f-8647e8039eee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:13.381526 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c9041-45f9-4756-9c7f-8647e8039eee admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:13.471295 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:13.507010 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a49dd-e105-41cb-81af-ba223bd09919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:13.509729 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a49dd-e105-41cb-81af-ba223bd09919 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:13.510145 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a49dd-e105-41cb-81af-ba223bd09919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:13.510430 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a49dd-e105-41cb-81af-ba223bd09919 admin admin] PATH_INFO: `/v3/projects/8a48b70e4365461c96e8da8e0537ada6` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:13.512396 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01a49dd-e105-41cb-81af-ba223bd09919 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a48b70e4365461c96e8da8e0537ada6)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:13.515849 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01a49dd-e105-41cb-81af-ba223bd09919 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:13.608613 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f01a49dd-e105-41cb-81af-ba223bd09919 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a48b70e4365461c96e8da8e0537ada6', 'request_id': 'req-f01a49dd-e105-41cb-81af-ba223bd09919'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:13.616248 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f01a49dd-e105-41cb-81af-ba223bd09919 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a48b70e4365461c96e8da8e0537ada6', 'request_id': 'req-f01a49dd-e105-41cb-81af-ba223bd09919'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:13.632249 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f01a49dd-e105-41cb-81af-ba223bd09919 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a48b70e4365461c96e8da8e0537ada6', 'request_id': 'req-f01a49dd-e105-41cb-81af-ba223bd09919'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:13.691253 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f01a49dd-e105-41cb-81af-ba223bd09919 admin admin] The token cache is being invalidate because project 8a48b70e4365461c96e8da8e0537ada6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:30:13.691904 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f01a49dd-e105-41cb-81af-ba223bd09919 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:13.692920 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f01a49dd-e105-41cb-81af-ba223bd09919 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:13.696281 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1180/2336] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:30:13 2022] DELETE /identity/v3/projects/8a48b70e4365461c96e8da8e0537ada6 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:13.702897 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f4795-ff9d-4e1c-8414-e58b305006f4 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:13.703165 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f4795-ff9d-4e1c-8414-e58b305006f4 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:13.703449 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f4795-ff9d-4e1c-8414-e58b305006f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:13.728154 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-1c0f4795-ff9d-4e1c-8414-e58b305006f4 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:13.730397 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-1c0f4795-ff9d-4e1c-8414-e58b305006f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:13.843988 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1157/2337] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:13.861278 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26574045-b9d1-4078-8b7d-781437f406a7 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:13.861947 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26574045-b9d1-4078-8b7d-781437f406a7 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:13.862457 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26574045-b9d1-4078-8b7d-781437f406a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:13.903870 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-26574045-b9d1-4078-8b7d-781437f406a7 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:13.906953 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-26574045-b9d1-4078-8b7d-781437f406a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:13.924066 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1181/2338] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:13.929043 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f4795-ff9d-4e1c-8414-e58b305006f4 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:13.941808 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f4795-ff9d-4e1c-8414-e58b305006f4 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:13.946861 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24345bc8-93f8-4c45-878d-9d5459d50d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:13.947673 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24345bc8-93f8-4c45-878d-9d5459d50d4c admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:13.948331 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24345bc8-93f8-4c45-878d-9d5459d50d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:13.948710 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24345bc8-93f8-4c45-878d-9d5459d50d4c admin admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:13.949616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24345bc8-93f8-4c45-878d-9d5459d50d4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:13.950037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24345bc8-93f8-4c45-878d-9d5459d50d4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:13.952023 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24345bc8-93f8-4c45-878d-9d5459d50d4c admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:13.975687 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1158/2339] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:30:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:13.979943 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26574045-b9d1-4078-8b7d-781437f406a7 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:13.989957 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26574045-b9d1-4078-8b7d-781437f406a7 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:13.998141 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec752ec-df58-44cb-9563-1c2b44a1e347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:13.998826 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec752ec-df58-44cb-9563-1c2b44a1e347 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:13.999106 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec752ec-df58-44cb-9563-1c2b44a1e347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:13.999378 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec752ec-df58-44cb-9563-1c2b44a1e347 admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:14.000077 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec752ec-df58-44cb-9563-1c2b44a1e347 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:14.002061 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec752ec-df58-44cb-9563-1c2b44a1e347 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:14.036150 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1182/2340] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:30:13 2022] POST /identity/v3/projects => generated 393 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:14.040749 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24345bc8-93f8-4c45-878d-9d5459d50d4c admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:14.057480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24345bc8-93f8-4c45-878d-9d5459d50d4c admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:14.130513 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beabb49-9b0a-4621-88f8-ee19e4f1eaf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:14.131604 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beabb49-9b0a-4621-88f8-ee19e4f1eaf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:14.132389 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beabb49-9b0a-4621-88f8-ee19e4f1eaf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:14.132824 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beabb49-9b0a-4621-88f8-ee19e4f1eaf7 admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:14.133710 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beabb49-9b0a-4621-88f8-ee19e4f1eaf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:14.136038 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beabb49-9b0a-4621-88f8-ee19e4f1eaf7 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:14.166531 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1159/2341] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:30:14 2022] POST /identity/v3/users => generated 297 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:14.167596 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec752ec-df58-44cb-9563-1c2b44a1e347 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:14.183197 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec752ec-df58-44cb-9563-1c2b44a1e347 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:14.193960 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a8fa3-1ac6-40e1-b836-204211a23334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:14.195148 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a8fa3-1ac6-40e1-b836-204211a23334 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:14.195430 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a8fa3-1ac6-40e1-b836-204211a23334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:14.195708 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0a8fa3-1ac6-40e1-b836-204211a23334 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:14.196445 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0a8fa3-1ac6-40e1-b836-204211a23334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:14.198306 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0a8fa3-1ac6-40e1-b836-204211a23334 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:14.212992 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1183/2342] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:14.215383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beabb49-9b0a-4621-88f8-ee19e4f1eaf7 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:14.226345 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beabb49-9b0a-4621-88f8-ee19e4f1eaf7 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:14.231255 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902753fd-0976-4328-b9db-7f29b0ca6daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:14.232045 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902753fd-0976-4328-b9db-7f29b0ca6daf admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:14.232426 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902753fd-0976-4328-b9db-7f29b0ca6daf admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:14.232792 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902753fd-0976-4328-b9db-7f29b0ca6daf admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:14.233916 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902753fd-0976-4328-b9db-7f29b0ca6daf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:14.235618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902753fd-0976-4328-b9db-7f29b0ca6daf admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:14.253024 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1160/2343] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:30:14.254680 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a8fa3-1ac6-40e1-b836-204211a23334 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:14.272707 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0a8fa3-1ac6-40e1-b836-204211a23334 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:14.277671 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90750fc-c88e-444d-af00-3e13b695a48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:14.278695 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90750fc-c88e-444d-af00-3e13b695a48d admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:14.279039 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90750fc-c88e-444d-af00-3e13b695a48d admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:14.279384 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90750fc-c88e-444d-af00-3e13b695a48d admin admin] PATH_INFO: `/v3/projects/66d8574f90784c13b23722c02e0f8e9f/users/de32bb6316c948a98b85d54515031eb1/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:14.299187 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90750fc-c88e-444d-af00-3e13b695a48d admin admin] RBAC: Authorizing `identity:create_grant(project_id=66d8574f90784c13b23722c02e0f8e9f, user_id=de32bb6316c948a98b85d54515031eb1, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:14.302126 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90750fc-c88e-444d-af00-3e13b695a48d admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:14.320851 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1184/2344] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:30:14 2022] PUT /identity/v3/projects/66d8574f90784c13b23722c02e0f8e9f/users/de32bb6316c948a98b85d54515031eb1/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:14.331442 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c13ea7-ecab-449e-88f7-158b9a5079ee None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:14.331735 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c13ea7-ecab-449e-88f7-158b9a5079ee None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:14.332013 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c13ea7-ecab-449e-88f7-158b9a5079ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:14.344832 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:14.361834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-29c13ea7-ecab-449e-88f7-158b9a5079ee None None] MFA Rules not processed for user `de32bb6316c948a98b85d54515031eb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:14.363067 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-29c13ea7-ecab-449e-88f7-158b9a5079ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:14.434213 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1161/2345] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:30:14.438658 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90750fc-c88e-444d-af00-3e13b695a48d admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:14.499744 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90750fc-c88e-444d-af00-3e13b695a48d admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:14.504715 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c221b-5e1b-4b90-ba61-626b8b3b2b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:14.505459 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c221b-5e1b-4b90-ba61-626b8b3b2b61 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:14.505732 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c221b-5e1b-4b90-ba61-626b8b3b2b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:14.506029 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c221b-5e1b-4b90-ba61-626b8b3b2b61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:14.507199 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71c221b-5e1b-4b90-ba61-626b8b3b2b61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:14.509177 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71c221b-5e1b-4b90-ba61-626b8b3b2b61 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:14.522972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:14.540833 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1185/2346] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:30:14 2022] POST /identity/v3/projects => generated 393 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:14.544865 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c13ea7-ecab-449e-88f7-158b9a5079ee None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:14.556819 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c13ea7-ecab-449e-88f7-158b9a5079ee None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:14.617402 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbda0f02-23f2-473d-aff9-cbc2c067643f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:14.618142 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda0f02-23f2-473d-aff9-cbc2c067643f admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:14.618421 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda0f02-23f2-473d-aff9-cbc2c067643f admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:14.618698 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda0f02-23f2-473d-aff9-cbc2c067643f admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:14.619392 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbda0f02-23f2-473d-aff9-cbc2c067643f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:14.621356 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbda0f02-23f2-473d-aff9-cbc2c067643f admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:14.643456 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1162/2347] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:30:14 2022] POST /identity/v3/users => generated 303 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:14.647738 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c221b-5e1b-4b90-ba61-626b8b3b2b61 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:14.658173 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c221b-5e1b-4b90-ba61-626b8b3b2b61 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:14.663612 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8900051-a992-48e3-a3c0-5ced69060460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:14.664273 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8900051-a992-48e3-a3c0-5ced69060460 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:14.664550 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8900051-a992-48e3-a3c0-5ced69060460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:14.664823 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8900051-a992-48e3-a3c0-5ced69060460 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:14.665562 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8900051-a992-48e3-a3c0-5ced69060460 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:14.667185 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8900051-a992-48e3-a3c0-5ced69060460 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:14.677750 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1186/2348] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:14.682580 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbda0f02-23f2-473d-aff9-cbc2c067643f admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:14.692873 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbda0f02-23f2-473d-aff9-cbc2c067643f admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:14.698149 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727101d2-f6c9-4bcc-ba7b-cf6697671183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:14.699033 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727101d2-f6c9-4bcc-ba7b-cf6697671183 admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:14.699306 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727101d2-f6c9-4bcc-ba7b-cf6697671183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:14.699595 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727101d2-f6c9-4bcc-ba7b-cf6697671183 admin admin] PATH_INFO: `/v3/projects/edf22c4c22144daa89191412302d85ab/users/415027c512194dc5b3c324734f06f895/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:14.716693 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727101d2-f6c9-4bcc-ba7b-cf6697671183 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edf22c4c22144daa89191412302d85ab, user_id=415027c512194dc5b3c324734f06f895, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:14.719423 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727101d2-f6c9-4bcc-ba7b-cf6697671183 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:14.734266 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1163/2349] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:30:14 2022] PUT /identity/v3/projects/edf22c4c22144daa89191412302d85ab/users/415027c512194dc5b3c324734f06f895/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:14.738500 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8900051-a992-48e3-a3c0-5ced69060460 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:14.749484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8900051-a992-48e3-a3c0-5ced69060460 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:14.808373 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3468d474-51c4-49ae-a2ef-e57953a2ea3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:14.809113 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468d474-51c4-49ae-a2ef-e57953a2ea3d admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:14.809388 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468d474-51c4-49ae-a2ef-e57953a2ea3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:14.809659 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3468d474-51c4-49ae-a2ef-e57953a2ea3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:14.810401 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3468d474-51c4-49ae-a2ef-e57953a2ea3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:14.812040 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3468d474-51c4-49ae-a2ef-e57953a2ea3d admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:14.823825 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1187/2350] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:14.828087 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727101d2-f6c9-4bcc-ba7b-cf6697671183 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:14.839208 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727101d2-f6c9-4bcc-ba7b-cf6697671183 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:14.843752 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62659da0-64fa-4bf0-bb6c-f08dcdd7117e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:14.844637 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62659da0-64fa-4bf0-bb6c-f08dcdd7117e admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:14.844912 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62659da0-64fa-4bf0-bb6c-f08dcdd7117e admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:14.845209 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62659da0-64fa-4bf0-bb6c-f08dcdd7117e admin admin] PATH_INFO: `/v3/projects/edf22c4c22144daa89191412302d85ab/users/415027c512194dc5b3c324734f06f895/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:14.847289 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62659da0-64fa-4bf0-bb6c-f08dcdd7117e admin admin] RBAC: Authorizing `identity:create_grant(project_id=edf22c4c22144daa89191412302d85ab, user_id=415027c512194dc5b3c324734f06f895, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:14.850175 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62659da0-64fa-4bf0-bb6c-f08dcdd7117e admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:14.860877 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1164/2351] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:30:14 2022] PUT /identity/v3/projects/edf22c4c22144daa89191412302d85ab/users/415027c512194dc5b3c324734f06f895/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:14.871372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287143f-043a-4fb0-bec5-3515df9d4762 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:14.871667 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287143f-043a-4fb0-bec5-3515df9d4762 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:14.871949 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287143f-043a-4fb0-bec5-3515df9d4762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:14.899385 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-d287143f-043a-4fb0-bec5-3515df9d4762 None None] MFA Rules not processed for user `415027c512194dc5b3c324734f06f895`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:14.900590 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-d287143f-043a-4fb0-bec5-3515df9d4762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:14.973785 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1188/2352] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:14 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:14.989086 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62659da0-64fa-4bf0-bb6c-f08dcdd7117e admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:14.991015 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-62659da0-64fa-4bf0-bb6c-f08dcdd7117e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:15.046113 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:15.050463 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62659da0-64fa-4bf0-bb6c-f08dcdd7117e admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:15.055577 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357bab8-a1e3-4c57-9b28-b22be612fc45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:15.056234 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357bab8-a1e3-4c57-9b28-b22be612fc45 service cinder] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:15.056522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357bab8-a1e3-4c57-9b28-b22be612fc45 service cinder] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:15.056802 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357bab8-a1e3-4c57-9b28-b22be612fc45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:15.066697 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9357bab8-a1e3-4c57-9b28-b22be612fc45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:15.068288 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9357bab8-a1e3-4c57-9b28-b22be612fc45 service cinder] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:15.078914 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1165/2353] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:30:14 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:30:15.131610 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287143f-043a-4fb0-bec5-3515df9d4762 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:15.134112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-d287143f-043a-4fb0-bec5-3515df9d4762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:15.151184 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287143f-043a-4fb0-bec5-3515df9d4762 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:15.155877 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f527ab97-cec0-4838-9a49-d81811a2f6a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:15.156572 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527ab97-cec0-4838-9a49-d81811a2f6a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:15.156855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527ab97-cec0-4838-9a49-d81811a2f6a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:15.157168 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f527ab97-cec0-4838-9a49-d81811a2f6a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:15.217604 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f527ab97-cec0-4838-9a49-d81811a2f6a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:15.219065 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f527ab97-cec0-4838-9a49-d81811a2f6a7 service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:15.228586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1189/2354] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:30:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:30:18.771058 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357bab8-a1e3-4c57-9b28-b22be612fc45 service cinder] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:18.872584 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357bab8-a1e3-4c57-9b28-b22be612fc45 service cinder] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:18.878043 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e2ce11-de87-47ab-97ad-5d90f877bda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:18.879134 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e2ce11-de87-47ab-97ad-5d90f877bda1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:18.879723 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e2ce11-de87-47ab-97ad-5d90f877bda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:18.880200 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e2ce11-de87-47ab-97ad-5d90f877bda1 admin admin] PATH_INFO: `/v3/users/de32bb6316c948a98b85d54515031eb1` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:18.881998 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e2ce11-de87-47ab-97ad-5d90f877bda1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de32bb6316c948a98b85d54515031eb1)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:18.884502 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e2ce11-de87-47ab-97ad-5d90f877bda1 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:18.940174 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-a9e2ce11-de87-47ab-97ad-5d90f877bda1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de32bb6316c948a98b85d54515031eb1'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:18.952906 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-a9e2ce11-de87-47ab-97ad-5d90f877bda1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de32bb6316c948a98b85d54515031eb1'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:18.963984 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-a9e2ce11-de87-47ab-97ad-5d90f877bda1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de32bb6316c948a98b85d54515031eb1'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:18.975690 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-a9e2ce11-de87-47ab-97ad-5d90f877bda1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de32bb6316c948a98b85d54515031eb1'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:18.977388 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-a9e2ce11-de87-47ab-97ad-5d90f877bda1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de32bb6316c948a98b85d54515031eb1'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:18.984941 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1166/2355] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:30:18 2022] DELETE /identity/v3/users/de32bb6316c948a98b85d54515031eb1 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:18.989322 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99d892-404d-453d-b122-acef7c58208e None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:18.991040 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99d892-404d-453d-b122-acef7c58208e None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:18.993354 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a99d892-404d-453d-b122-acef7c58208e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:19.004982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:19.032079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-4a99d892-404d-453d-b122-acef7c58208e None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:19.032079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-4a99d892-404d-453d-b122-acef7c58208e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:19.120104 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1190/2356] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:19.137357 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e2ce11-de87-47ab-97ad-5d90f877bda1 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:19.138925 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-a9e2ce11-de87-47ab-97ad-5d90f877bda1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:19.205704 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e2ce11-de87-47ab-97ad-5d90f877bda1 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:19.209651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a91bcd-2953-45e0-b447-932506838a7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:19.210475 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a91bcd-2953-45e0-b447-932506838a7e service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:19.210756 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a91bcd-2953-45e0-b447-932506838a7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:19.211030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a91bcd-2953-45e0-b447-932506838a7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:19.221774 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a91bcd-2953-45e0-b447-932506838a7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:19.223365 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a91bcd-2953-45e0-b447-932506838a7e service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:19.234873 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1167/2357] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:30:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:30:19.513377 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99d892-404d-453d-b122-acef7c58208e None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:19.514910 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-4a99d892-404d-453d-b122-acef7c58208e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:19.529985 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a99d892-404d-453d-b122-acef7c58208e None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:19.535350 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e42577-13b0-4c25-8d35-f6b58e5f4dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:19.536223 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e42577-13b0-4c25-8d35-f6b58e5f4dff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:19.536511 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e42577-13b0-4c25-8d35-f6b58e5f4dff admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:19.536788 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e42577-13b0-4c25-8d35-f6b58e5f4dff admin admin] PATH_INFO: `/v3/projects/66d8574f90784c13b23722c02e0f8e9f` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:19.538774 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e42577-13b0-4c25-8d35-f6b58e5f4dff admin admin] RBAC: Authorizing `identity:delete_project(project_id=66d8574f90784c13b23722c02e0f8e9f)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:19.540925 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e42577-13b0-4c25-8d35-f6b58e5f4dff admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:19.590860 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-b2e42577-13b0-4c25-8d35-f6b58e5f4dff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66d8574f90784c13b23722c02e0f8e9f', 'request_id': 'req-b2e42577-13b0-4c25-8d35-f6b58e5f4dff'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:19.598030 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-b2e42577-13b0-4c25-8d35-f6b58e5f4dff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66d8574f90784c13b23722c02e0f8e9f', 'request_id': 'req-b2e42577-13b0-4c25-8d35-f6b58e5f4dff'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:19.606827 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-b2e42577-13b0-4c25-8d35-f6b58e5f4dff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66d8574f90784c13b23722c02e0f8e9f', 'request_id': 'req-b2e42577-13b0-4c25-8d35-f6b58e5f4dff'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:19.621608 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-b2e42577-13b0-4c25-8d35-f6b58e5f4dff admin admin] The token cache is being invalidate because project 66d8574f90784c13b23722c02e0f8e9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:30:19.621891 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-b2e42577-13b0-4c25-8d35-f6b58e5f4dff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:19.622503 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-b2e42577-13b0-4c25-8d35-f6b58e5f4dff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:19.624087 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1191/2358] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:30:19 2022] DELETE /identity/v3/projects/66d8574f90784c13b23722c02e0f8e9f => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:19.628121 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a91bcd-2953-45e0-b447-932506838a7e service neutron] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:19.638591 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a91bcd-2953-45e0-b447-932506838a7e service neutron] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:19.648116 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:19.706682 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a9bb9d-744b-460a-b5aa-f65fe935adb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:19.707571 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9bb9d-744b-460a-b5aa-f65fe935adb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:19.707851 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9bb9d-744b-460a-b5aa-f65fe935adb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:19.708115 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9bb9d-744b-460a-b5aa-f65fe935adb2 admin admin] PATH_INFO: `/v3/users/415027c512194dc5b3c324734f06f895` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:19.710077 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a9bb9d-744b-460a-b5aa-f65fe935adb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=415027c512194dc5b3c324734f06f895)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:19.712576 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a9bb9d-744b-460a-b5aa-f65fe935adb2 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:19.771519 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-75a9bb9d-744b-460a-b5aa-f65fe935adb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '415027c512194dc5b3c324734f06f895'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:19.787119 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-75a9bb9d-744b-460a-b5aa-f65fe935adb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '415027c512194dc5b3c324734f06f895'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:19.800662 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-75a9bb9d-744b-460a-b5aa-f65fe935adb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '415027c512194dc5b3c324734f06f895'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:19.811272 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-75a9bb9d-744b-460a-b5aa-f65fe935adb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '415027c512194dc5b3c324734f06f895'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:19.811918 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-75a9bb9d-744b-460a-b5aa-f65fe935adb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '415027c512194dc5b3c324734f06f895'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:19.823350 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1168/2359] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:30:19 2022] DELETE /identity/v3/users/415027c512194dc5b3c324734f06f895 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:20.258124 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e42577-13b0-4c25-8d35-f6b58e5f4dff admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:20.259665 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-b2e42577-13b0-4c25-8d35-f6b58e5f4dff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:20.283820 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e42577-13b0-4c25-8d35-f6b58e5f4dff admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:20.390754 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3e5569-7e98-4ed1-8b85-0db5b121911f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:20.391936 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3e5569-7e98-4ed1-8b85-0db5b121911f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:20.392553 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3e5569-7e98-4ed1-8b85-0db5b121911f admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:20.392846 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3e5569-7e98-4ed1-8b85-0db5b121911f admin admin] PATH_INFO: `/v3/projects/edf22c4c22144daa89191412302d85ab` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:20.394350 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3e5569-7e98-4ed1-8b85-0db5b121911f admin admin] RBAC: Authorizing `identity:delete_project(project_id=edf22c4c22144daa89191412302d85ab)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:20.396750 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3e5569-7e98-4ed1-8b85-0db5b121911f admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:20.434604 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:20.477929 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3d3e5569-7e98-4ed1-8b85-0db5b121911f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edf22c4c22144daa89191412302d85ab', 'request_id': 'req-3d3e5569-7e98-4ed1-8b85-0db5b121911f'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:20.489972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3d3e5569-7e98-4ed1-8b85-0db5b121911f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edf22c4c22144daa89191412302d85ab', 'request_id': 'req-3d3e5569-7e98-4ed1-8b85-0db5b121911f'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:20.502052 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3d3e5569-7e98-4ed1-8b85-0db5b121911f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edf22c4c22144daa89191412302d85ab', 'request_id': 'req-3d3e5569-7e98-4ed1-8b85-0db5b121911f'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:20.514833 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3d3e5569-7e98-4ed1-8b85-0db5b121911f admin admin] The token cache is being invalidate because project edf22c4c22144daa89191412302d85ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:30:20.515141 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3d3e5569-7e98-4ed1-8b85-0db5b121911f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:20.515788 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3d3e5569-7e98-4ed1-8b85-0db5b121911f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:20.517422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1192/2360] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:30:20 2022] DELETE /identity/v3/projects/edf22c4c22144daa89191412302d85ab => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:20.524686 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7e257-39cf-45b7-8489-a14a219f58a8 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:20.528128 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7e257-39cf-45b7-8489-a14a219f58a8 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:20.528128 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7e257-39cf-45b7-8489-a14a219f58a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:20.546374 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-10f7e257-39cf-45b7-8489-a14a219f58a8 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:20.547597 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-10f7e257-39cf-45b7-8489-a14a219f58a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:20.624596 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1169/2361] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:20.636596 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b6553-d67b-4a4b-bbf9-87080360cc2e None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:20.636787 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b6553-d67b-4a4b-bbf9-87080360cc2e None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:20.637136 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b6553-d67b-4a4b-bbf9-87080360cc2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:20.657202 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-2a3b6553-d67b-4a4b-bbf9-87080360cc2e None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:20.658480 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-2a3b6553-d67b-4a4b-bbf9-87080360cc2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:20.673999 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1193/2362] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:20.678567 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f7e257-39cf-45b7-8489-a14a219f58a8 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:20.689234 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f7e257-39cf-45b7-8489-a14a219f58a8 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:20.694604 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424e5d6b-2356-444a-bf8e-c4a1fc1e1461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:20.695287 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e5d6b-2356-444a-bf8e-c4a1fc1e1461 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:20.695577 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e5d6b-2356-444a-bf8e-c4a1fc1e1461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:20.695852 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e5d6b-2356-444a-bf8e-c4a1fc1e1461 admin admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:20.696602 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424e5d6b-2356-444a-bf8e-c4a1fc1e1461 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:20.696901 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424e5d6b-2356-444a-bf8e-c4a1fc1e1461 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:20.698571 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424e5d6b-2356-444a-bf8e-c4a1fc1e1461 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:20.716705 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1170/2363] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:30:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:30:20.721288 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3b6553-d67b-4a4b-bbf9-87080360cc2e None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:20.732243 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3b6553-d67b-4a4b-bbf9-87080360cc2e None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:20.736937 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc654c1-4d94-4420-9044-e5305fea1129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:20.737619 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc654c1-4d94-4420-9044-e5305fea1129 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:20.737902 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc654c1-4d94-4420-9044-e5305fea1129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:20.738426 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc654c1-4d94-4420-9044-e5305fea1129 admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:20.739164 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc654c1-4d94-4420-9044-e5305fea1129 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:20.741333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc654c1-4d94-4420-9044-e5305fea1129 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:20.772989 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1194/2364] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:30:20 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:30:20.774911 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424e5d6b-2356-444a-bf8e-c4a1fc1e1461 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:20.785316 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424e5d6b-2356-444a-bf8e-c4a1fc1e1461 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:20.806320 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:20.852816 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0a33e8-e841-4a74-ac7d-c79d07cfa3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:20.853539 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a33e8-e841-4a74-ac7d-c79d07cfa3c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:20.853825 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a33e8-e841-4a74-ac7d-c79d07cfa3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:20.854229 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a33e8-e841-4a74-ac7d-c79d07cfa3c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:20.854975 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0a33e8-e841-4a74-ac7d-c79d07cfa3c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:20.856946 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0a33e8-e841-4a74-ac7d-c79d07cfa3c8 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:20.878760 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1171/2365] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:30:20 2022] POST /identity/v3/users => generated 290 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:20.883903 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc654c1-4d94-4420-9044-e5305fea1129 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:20.894643 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc654c1-4d94-4420-9044-e5305fea1129 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:20.899352 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355875d9-6416-4f29-9d33-2293a9023245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:20.900022 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355875d9-6416-4f29-9d33-2293a9023245 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:20.900300 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355875d9-6416-4f29-9d33-2293a9023245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:20.900592 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355875d9-6416-4f29-9d33-2293a9023245 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:20.901492 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355875d9-6416-4f29-9d33-2293a9023245 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:20.903111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355875d9-6416-4f29-9d33-2293a9023245 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:20.914053 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1195/2366] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:30:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:30:20.918286 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0a33e8-e841-4a74-ac7d-c79d07cfa3c8 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:20.928386 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0a33e8-e841-4a74-ac7d-c79d07cfa3c8 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:20.933315 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5580d98-0a7a-4e3f-a5ad-ff07812a81fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:20.933980 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5580d98-0a7a-4e3f-a5ad-ff07812a81fe admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:20.934294 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5580d98-0a7a-4e3f-a5ad-ff07812a81fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:20.934565 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5580d98-0a7a-4e3f-a5ad-ff07812a81fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:20.935286 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5580d98-0a7a-4e3f-a5ad-ff07812a81fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:20.936919 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5580d98-0a7a-4e3f-a5ad-ff07812a81fe admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:20.949214 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1172/2367] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:30:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:20.953429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355875d9-6416-4f29-9d33-2293a9023245 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:20.964282 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355875d9-6416-4f29-9d33-2293a9023245 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:20.969097 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f601905e-5075-4566-9a7b-16fd6c104773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:20.969984 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601905e-5075-4566-9a7b-16fd6c104773 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:20.970260 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601905e-5075-4566-9a7b-16fd6c104773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:20.970533 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601905e-5075-4566-9a7b-16fd6c104773 admin admin] PATH_INFO: `/v3/projects/4c1ffbd612d04ad59762f23366fd816d/users/ccd9583fb7a24319ba00dffdbe0ad5a3/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:20.985932 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f601905e-5075-4566-9a7b-16fd6c104773 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c1ffbd612d04ad59762f23366fd816d, user_id=ccd9583fb7a24319ba00dffdbe0ad5a3, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:20.988623 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f601905e-5075-4566-9a7b-16fd6c104773 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:21.005516 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1196/2368] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:30:20 2022] PUT /identity/v3/projects/4c1ffbd612d04ad59762f23366fd816d/users/ccd9583fb7a24319ba00dffdbe0ad5a3/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:21.021700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab975c-665b-4a3b-bcdf-2bead36c7eda None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:21.022012 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab975c-665b-4a3b-bcdf-2bead36c7eda None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:21.022263 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ab975c-665b-4a3b-bcdf-2bead36c7eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:21.051356 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-21ab975c-665b-4a3b-bcdf-2bead36c7eda None None] MFA Rules not processed for user `ccd9583fb7a24319ba00dffdbe0ad5a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:21.052595 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-21ab975c-665b-4a3b-bcdf-2bead36c7eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:21.126208 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1173/2369] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:21.131087 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f601905e-5075-4566-9a7b-16fd6c104773 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:21.197702 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f601905e-5075-4566-9a7b-16fd6c104773 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:21.204172 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e2167-b15e-4a6d-b30a-e0cbf25e3d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:21.205966 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e2167-b15e-4a6d-b30a-e0cbf25e3d99 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:21.206262 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e2167-b15e-4a6d-b30a-e0cbf25e3d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:21.206556 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e2167-b15e-4a6d-b30a-e0cbf25e3d99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:21.207272 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48e2167-b15e-4a6d-b30a-e0cbf25e3d99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:21.209389 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48e2167-b15e-4a6d-b30a-e0cbf25e3d99 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:21.237652 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1197/2370] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:30:21 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:30:21.241919 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ab975c-665b-4a3b-bcdf-2bead36c7eda None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:21.252593 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ab975c-665b-4a3b-bcdf-2bead36c7eda None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:21.312411 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2de291-560f-42e6-817b-6c548bbab8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:21.313204 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2de291-560f-42e6-817b-6c548bbab8c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:21.313483 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2de291-560f-42e6-817b-6c548bbab8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:21.313747 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2de291-560f-42e6-817b-6c548bbab8c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:21.314452 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2de291-560f-42e6-817b-6c548bbab8c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:21.316372 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2de291-560f-42e6-817b-6c548bbab8c3 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:21.340749 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1174/2371] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:30:21 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:30:21.342078 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e2167-b15e-4a6d-b30a-e0cbf25e3d99 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:21.355650 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e2167-b15e-4a6d-b30a-e0cbf25e3d99 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:21.361261 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d71a15a-149d-44c1-b1b3-0f2b51cc2e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:21.361930 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71a15a-149d-44c1-b1b3-0f2b51cc2e08 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:21.362187 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71a15a-149d-44c1-b1b3-0f2b51cc2e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:21.362480 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71a15a-149d-44c1-b1b3-0f2b51cc2e08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:21.363198 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d71a15a-149d-44c1-b1b3-0f2b51cc2e08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:21.364823 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d71a15a-149d-44c1-b1b3-0f2b51cc2e08 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:21.376258 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1198/2372] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:30:21.380734 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2de291-560f-42e6-817b-6c548bbab8c3 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:21.392422 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2de291-560f-42e6-817b-6c548bbab8c3 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:21.398184 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef07b96e-8df9-4c99-a966-4df82ecd55a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:21.398972 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07b96e-8df9-4c99-a966-4df82ecd55a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:21.399257 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07b96e-8df9-4c99-a966-4df82ecd55a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:21.399534 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef07b96e-8df9-4c99-a966-4df82ecd55a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:21.400269 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07b96e-8df9-4c99-a966-4df82ecd55a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:21.401950 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef07b96e-8df9-4c99-a966-4df82ecd55a9 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:21.413476 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1175/2373] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:21.417919 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d71a15a-149d-44c1-b1b3-0f2b51cc2e08 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:21.428773 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d71a15a-149d-44c1-b1b3-0f2b51cc2e08 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:21.433681 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c5004c-7b16-4960-ac77-e45a62a4c9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:21.434694 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5004c-7b16-4960-ac77-e45a62a4c9c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:21.434903 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5004c-7b16-4960-ac77-e45a62a4c9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:21.435176 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5004c-7b16-4960-ac77-e45a62a4c9c7 admin admin] PATH_INFO: `/v3/projects/c92e403fe66644db94aa3905f992b420/users/a176a9c303b749278da00e16887ecd6b/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:21.453535 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c5004c-7b16-4960-ac77-e45a62a4c9c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c92e403fe66644db94aa3905f992b420, user_id=a176a9c303b749278da00e16887ecd6b, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:21.457415 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c5004c-7b16-4960-ac77-e45a62a4c9c7 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:21.471354 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1199/2374] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:30:21 2022] PUT /identity/v3/projects/c92e403fe66644db94aa3905f992b420/users/a176a9c303b749278da00e16887ecd6b/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:21.485176 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c657a-4ed7-45da-8c87-5d1099db5e8e None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:21.485594 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c657a-4ed7-45da-8c87-5d1099db5e8e None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:21.485766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0c657a-4ed7-45da-8c87-5d1099db5e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:21.501487 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:21.521414 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-9c0c657a-4ed7-45da-8c87-5d1099db5e8e None None] MFA Rules not processed for user `a176a9c303b749278da00e16887ecd6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:21.522959 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-9c0c657a-4ed7-45da-8c87-5d1099db5e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:21.601145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1176/2375] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:21.605540 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c5004c-7b16-4960-ac77-e45a62a4c9c7 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:21.610788 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:21.675515 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c5004c-7b16-4960-ac77-e45a62a4c9c7 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:21.680993 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e86577f-c5a0-46fc-ac1f-c04b188103ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:21.681573 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86577f-c5a0-46fc-ac1f-c04b188103ca admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:21.681853 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86577f-c5a0-46fc-ac1f-c04b188103ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:21.682128 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e86577f-c5a0-46fc-ac1f-c04b188103ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:21.682835 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e86577f-c5a0-46fc-ac1f-c04b188103ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:21.684790 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e86577f-c5a0-46fc-ac1f-c04b188103ca admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:21.723730 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1200/2376] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:30:21 2022] POST /identity/v3/projects => generated 379 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:21.728377 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0c657a-4ed7-45da-8c87-5d1099db5e8e None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:21.739039 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0c657a-4ed7-45da-8c87-5d1099db5e8e None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:21.801507 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a241199-b65e-4c5e-8808-b3808fd1ef76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:21.802264 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a241199-b65e-4c5e-8808-b3808fd1ef76 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:21.802389 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a241199-b65e-4c5e-8808-b3808fd1ef76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:21.802898 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a241199-b65e-4c5e-8808-b3808fd1ef76 admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:21.803998 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a241199-b65e-4c5e-8808-b3808fd1ef76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:21.805993 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a241199-b65e-4c5e-8808-b3808fd1ef76 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:21.830913 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1177/2377] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:30:21 2022] POST /identity/v3/users => generated 296 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:21.832177 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e86577f-c5a0-46fc-ac1f-c04b188103ca admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:21.843924 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e86577f-c5a0-46fc-ac1f-c04b188103ca admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:21.849322 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d302b-d4e5-4387-bec4-dc57decfcc5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:21.850243 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d302b-d4e5-4387-bec4-dc57decfcc5d admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:21.850525 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d302b-d4e5-4387-bec4-dc57decfcc5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:21.850797 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d302b-d4e5-4387-bec4-dc57decfcc5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:21.851560 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847d302b-d4e5-4387-bec4-dc57decfcc5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:21.853225 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847d302b-d4e5-4387-bec4-dc57decfcc5d admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:21.864787 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1201/2378] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:21.869358 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a241199-b65e-4c5e-8808-b3808fd1ef76 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:21.879706 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a241199-b65e-4c5e-8808-b3808fd1ef76 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:21.885650 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a28fa4-4bc5-4c2b-810f-0173853119ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:21.886548 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a28fa4-4bc5-4c2b-810f-0173853119ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:21.886839 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a28fa4-4bc5-4c2b-810f-0173853119ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:21.887112 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a28fa4-4bc5-4c2b-810f-0173853119ca admin admin] PATH_INFO: `/v3/projects/5300ce567c4341a9a8854577e2c7a29d/users/3884f0125b604820b548db7564afb574/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:21.902422 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a28fa4-4bc5-4c2b-810f-0173853119ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=5300ce567c4341a9a8854577e2c7a29d, user_id=3884f0125b604820b548db7564afb574, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:21.905174 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a28fa4-4bc5-4c2b-810f-0173853119ca admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:21.919014 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1178/2379] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:30:21 2022] PUT /identity/v3/projects/5300ce567c4341a9a8854577e2c7a29d/users/3884f0125b604820b548db7564afb574/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:30:21.923186 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d302b-d4e5-4387-bec4-dc57decfcc5d admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:21.933852 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d302b-d4e5-4387-bec4-dc57decfcc5d admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:21.991807 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c73b46-5ee5-4b08-aa4a-1d68e56caf28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:21.992405 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c73b46-5ee5-4b08-aa4a-1d68e56caf28 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:21.992720 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c73b46-5ee5-4b08-aa4a-1d68e56caf28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:21.992976 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c73b46-5ee5-4b08-aa4a-1d68e56caf28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:21.993752 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c73b46-5ee5-4b08-aa4a-1d68e56caf28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:21.995380 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c73b46-5ee5-4b08-aa4a-1d68e56caf28 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:22.005313 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1202/2380] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:22.010725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a28fa4-4bc5-4c2b-810f-0173853119ca admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:22.021331 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a28fa4-4bc5-4c2b-810f-0173853119ca admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:22.026822 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b6d194-7010-4778-892a-15067a2a9873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:22.027719 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6d194-7010-4778-892a-15067a2a9873 admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:22.027998 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6d194-7010-4778-892a-15067a2a9873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:22.028275 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6d194-7010-4778-892a-15067a2a9873 admin admin] PATH_INFO: `/v3/projects/5300ce567c4341a9a8854577e2c7a29d/users/3884f0125b604820b548db7564afb574/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:22.030276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b6d194-7010-4778-892a-15067a2a9873 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5300ce567c4341a9a8854577e2c7a29d, user_id=3884f0125b604820b548db7564afb574, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:22.032937 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b6d194-7010-4778-892a-15067a2a9873 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:22.044640 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1179/2381] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:30:22 2022] PUT /identity/v3/projects/5300ce567c4341a9a8854577e2c7a29d/users/3884f0125b604820b548db7564afb574/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:22.055914 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8de8a-7f56-434a-b4c8-c2c5288c3f3c None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:22.056317 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8de8a-7f56-434a-b4c8-c2c5288c3f3c None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:22.056470 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8de8a-7f56-434a-b4c8-c2c5288c3f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:22.083331 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-c0f8de8a-7f56-434a-b4c8-c2c5288c3f3c None None] MFA Rules not processed for user `3884f0125b604820b548db7564afb574`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:22.084577 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-c0f8de8a-7f56-434a-b4c8-c2c5288c3f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:22.153153 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:22.159291 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1203/2382] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:22 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:30:22.171926 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b6d194-7010-4778-892a-15067a2a9873 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:22.174005 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-a5b6d194-7010-4778-892a-15067a2a9873 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:22.184399 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d2135-7e76-4e06-9686-bcd07f983399 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:22.184694 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d2135-7e76-4e06-9686-bcd07f983399 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:22.185001 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d2135-7e76-4e06-9686-bcd07f983399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:22.203498 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-7e7d2135-7e76-4e06-9686-bcd07f983399 None None] MFA Rules not processed for user `86a18e313e4944748aef58d54bfe7b13`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:22.204695 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-7e7d2135-7e76-4e06-9686-bcd07f983399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:22.234303 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:22.239816 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b6d194-7010-4778-892a-15067a2a9873 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:22.246416 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f572bc-5dd6-4fcd-9649-39d358f09cd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:22.247522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f572bc-5dd6-4fcd-9649-39d358f09cd6 service cinder] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:22.248609 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f572bc-5dd6-4fcd-9649-39d358f09cd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:22.248609 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f572bc-5dd6-4fcd-9649-39d358f09cd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:22.262572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f572bc-5dd6-4fcd-9649-39d358f09cd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:22.264285 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f572bc-5dd6-4fcd-9649-39d358f09cd6 service cinder] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:22.268625 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1204/2383] 10.222.0.50 () {58 vars in 967 bytes} [Fri May 27 03:30:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:30:22.282236 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1180/2384] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:30:22 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:30:22.282958 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7d2135-7e76-4e06-9686-bcd07f983399 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:22.284670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-7e7d2135-7e76-4e06-9686-bcd07f983399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:22.351805 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f572bc-5dd6-4fcd-9649-39d358f09cd6 service cinder] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:22.353334 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-a6f572bc-5dd6-4fcd-9649-39d358f09cd6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:22.364416 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f572bc-5dd6-4fcd-9649-39d358f09cd6 service cinder] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:22.371067 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7d2135-7e76-4e06-9686-bcd07f983399 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:22.371600 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd4f2a-9ba1-4583-9e30-4b2b4b597d4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:22.371943 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd4f2a-9ba1-4583-9e30-4b2b4b597d4f service cinder] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:22.372289 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd4f2a-9ba1-4583-9e30-4b2b4b597d4f service cinder] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:22.372630 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd4f2a-9ba1-4583-9e30-4b2b4b597d4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:22.376791 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c295f5ac-fb9f-4cba-b7f0-3ea570c0ad3e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de53e2f0fba4e0fac70887067c1963d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:22.377573 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295f5ac-fb9f-4cba-b7f0-3ea570c0ad3e service swift] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:22.377849 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295f5ac-fb9f-4cba-b7f0-3ea570c0ad3e service swift] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:22.378120 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295f5ac-fb9f-4cba-b7f0-3ea570c0ad3e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:22.388123 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c295f5ac-fb9f-4cba-b7f0-3ea570c0ad3e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:22.389883 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c295f5ac-fb9f-4cba-b7f0-3ea570c0ad3e service swift] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:22.399354 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1205/2385] 10.222.0.50 () {60 vars in 1384 bytes} [Fri May 27 03:30:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:30:22.461057 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd4f2a-9ba1-4583-9e30-4b2b4b597d4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:22.462456 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd4f2a-9ba1-4583-9e30-4b2b4b597d4f service cinder] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:22.472170 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1181/2386] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:30:22 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:30:22.765164 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c295f5ac-fb9f-4cba-b7f0-3ea570c0ad3e service swift] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:22.766625 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-c295f5ac-fb9f-4cba-b7f0-3ea570c0ad3e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:22.846687 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c295f5ac-fb9f-4cba-b7f0-3ea570c0ad3e service swift] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:22.851508 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9697a5-4ea8-4091-9bf1-63213d8b3f40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:22.852298 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9697a5-4ea8-4091-9bf1-63213d8b3f40 service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:22.852673 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9697a5-4ea8-4091-9bf1-63213d8b3f40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:22.853272 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9697a5-4ea8-4091-9bf1-63213d8b3f40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:22.856199 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-bc9697a5-4ea8-4091-9bf1-63213d8b3f40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:22.933428 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:22.952791 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9697a5-4ea8-4091-9bf1-63213d8b3f40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:22.954347 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9697a5-4ea8-4091-9bf1-63213d8b3f40 service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:22.965100 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1206/2387] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:30:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:30:25.600413 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd4f2a-9ba1-4583-9e30-4b2b4b597d4f service cinder] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:25.612079 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd4f2a-9ba1-4583-9e30-4b2b4b597d4f service cinder] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:25.617634 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe5586a-25ed-4715-914b-08d811c0ccd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:25.618766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe5586a-25ed-4715-914b-08d811c0ccd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:25.619053 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe5586a-25ed-4715-914b-08d811c0ccd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:25.619203 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe5586a-25ed-4715-914b-08d811c0ccd4 admin admin] PATH_INFO: `/v3/users/ccd9583fb7a24319ba00dffdbe0ad5a3` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:25.620555 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe5586a-25ed-4715-914b-08d811c0ccd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccd9583fb7a24319ba00dffdbe0ad5a3)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:25.622633 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe5586a-25ed-4715-914b-08d811c0ccd4 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:25.683324 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-2fe5586a-25ed-4715-914b-08d811c0ccd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccd9583fb7a24319ba00dffdbe0ad5a3'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:25.699240 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-2fe5586a-25ed-4715-914b-08d811c0ccd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccd9583fb7a24319ba00dffdbe0ad5a3'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:25.711072 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-2fe5586a-25ed-4715-914b-08d811c0ccd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccd9583fb7a24319ba00dffdbe0ad5a3'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:25.720617 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-2fe5586a-25ed-4715-914b-08d811c0ccd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccd9583fb7a24319ba00dffdbe0ad5a3'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:25.721603 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-2fe5586a-25ed-4715-914b-08d811c0ccd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccd9583fb7a24319ba00dffdbe0ad5a3'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:25.729846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1182/2388] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:30:25 2022] DELETE /identity/v3/users/ccd9583fb7a24319ba00dffdbe0ad5a3 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:25.734433 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac36bc99-b5fa-4ec4-8a4d-9f88e0884dbc None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:25.734933 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac36bc99-b5fa-4ec4-8a4d-9f88e0884dbc None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:25.735044 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac36bc99-b5fa-4ec4-8a4d-9f88e0884dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:25.753968 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-ac36bc99-b5fa-4ec4-8a4d-9f88e0884dbc None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:25.755331 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-ac36bc99-b5fa-4ec4-8a4d-9f88e0884dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:25.835609 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1207/2389] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:25.845917 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe5586a-25ed-4715-914b-08d811c0ccd4 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:25.849001 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-2fe5586a-25ed-4715-914b-08d811c0ccd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:25.948005 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe5586a-25ed-4715-914b-08d811c0ccd4 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:25.951882 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a2342-ef07-4bd8-8759-3a9a7b1f687f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:25.952549 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a2342-ef07-4bd8-8759-3a9a7b1f687f service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:25.952868 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a2342-ef07-4bd8-8759-3a9a7b1f687f service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:25.953138 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a2342-ef07-4bd8-8759-3a9a7b1f687f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:25.963608 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a2342-ef07-4bd8-8759-3a9a7b1f687f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:25.965557 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a2342-ef07-4bd8-8759-3a9a7b1f687f service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:25.976204 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1183/2390] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:30:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:30:26.544581 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac36bc99-b5fa-4ec4-8a4d-9f88e0884dbc None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:26.546264 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-ac36bc99-b5fa-4ec4-8a4d-9f88e0884dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:26.564730 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac36bc99-b5fa-4ec4-8a4d-9f88e0884dbc None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:26.571153 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b3ded-8cb5-4180-9ce5-88ffde91f1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:26.572152 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b3ded-8cb5-4180-9ce5-88ffde91f1e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:26.572498 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b3ded-8cb5-4180-9ce5-88ffde91f1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:26.572829 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b3ded-8cb5-4180-9ce5-88ffde91f1e3 admin admin] PATH_INFO: `/v3/projects/4c1ffbd612d04ad59762f23366fd816d` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:26.574269 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05b3ded-8cb5-4180-9ce5-88ffde91f1e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c1ffbd612d04ad59762f23366fd816d)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:26.576316 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05b3ded-8cb5-4180-9ce5-88ffde91f1e3 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:26.659437 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f05b3ded-8cb5-4180-9ce5-88ffde91f1e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c1ffbd612d04ad59762f23366fd816d', 'request_id': 'req-f05b3ded-8cb5-4180-9ce5-88ffde91f1e3'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:26.668761 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f05b3ded-8cb5-4180-9ce5-88ffde91f1e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c1ffbd612d04ad59762f23366fd816d', 'request_id': 'req-f05b3ded-8cb5-4180-9ce5-88ffde91f1e3'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:26.680781 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f05b3ded-8cb5-4180-9ce5-88ffde91f1e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c1ffbd612d04ad59762f23366fd816d', 'request_id': 'req-f05b3ded-8cb5-4180-9ce5-88ffde91f1e3'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:26.694433 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f05b3ded-8cb5-4180-9ce5-88ffde91f1e3 admin admin] The token cache is being invalidate because project 4c1ffbd612d04ad59762f23366fd816d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:30:26.694750 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f05b3ded-8cb5-4180-9ce5-88ffde91f1e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:26.695857 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f05b3ded-8cb5-4180-9ce5-88ffde91f1e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:26.700120 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1208/2391] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:30:26 2022] DELETE /identity/v3/projects/4c1ffbd612d04ad59762f23366fd816d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:26.701330 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a2342-ef07-4bd8-8759-3a9a7b1f687f service neutron] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:26.707672 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:26.712551 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a2342-ef07-4bd8-8759-3a9a7b1f687f service neutron] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:26.784728 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a06db0-a8cf-4039-b568-630e87928e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:26.785775 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a06db0-a8cf-4039-b568-630e87928e6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:26.786340 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a06db0-a8cf-4039-b568-630e87928e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:26.786615 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a06db0-a8cf-4039-b568-630e87928e6e admin admin] PATH_INFO: `/v3/users/a176a9c303b749278da00e16887ecd6b` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:26.787969 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b3ded-8cb5-4180-9ce5-88ffde91f1e3 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:26.788595 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a06db0-a8cf-4039-b568-630e87928e6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a176a9c303b749278da00e16887ecd6b)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:26.789422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-f05b3ded-8cb5-4180-9ce5-88ffde91f1e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:26.790523 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a06db0-a8cf-4039-b568-630e87928e6e admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:26.848755 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-09a06db0-a8cf-4039-b568-630e87928e6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a176a9c303b749278da00e16887ecd6b'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:26.868220 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-09a06db0-a8cf-4039-b568-630e87928e6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a176a9c303b749278da00e16887ecd6b'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:26.877054 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-09a06db0-a8cf-4039-b568-630e87928e6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a176a9c303b749278da00e16887ecd6b'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:26.879909 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b3ded-8cb5-4180-9ce5-88ffde91f1e3 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:26.884476 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-09a06db0-a8cf-4039-b568-630e87928e6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a176a9c303b749278da00e16887ecd6b'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:26.885127 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-09a06db0-a8cf-4039-b568-630e87928e6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a176a9c303b749278da00e16887ecd6b'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:26.886687 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df75fae-a360-45cf-a356-2a0d837761ac service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30963f2f464e4f54a8aea0175370a235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:26.887352 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df75fae-a360-45cf-a356-2a0d837761ac service placement] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:26.887622 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df75fae-a360-45cf-a356-2a0d837761ac service placement] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:26.887894 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df75fae-a360-45cf-a356-2a0d837761ac service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:26.889837 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-5df75fae-a360-45cf-a356-2a0d837761ac service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:26.893081 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1184/2392] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:30:26 2022] DELETE /identity/v3/users/a176a9c303b749278da00e16887ecd6b => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:30:26.893081 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a06db0-a8cf-4039-b568-630e87928e6e admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:26.894068 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-09a06db0-a8cf-4039-b568-630e87928e6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:26.906091 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:26.908663 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df75fae-a360-45cf-a356-2a0d837761ac service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:26.910891 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df75fae-a360-45cf-a356-2a0d837761ac service placement] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:26.924166 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1209/2393] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:30:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:30:26.987296 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a06db0-a8cf-4039-b568-630e87928e6e admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:26.994136 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a506e9-c07c-49a1-9f8d-8eb3f25d9e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:26.994917 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a506e9-c07c-49a1-9f8d-8eb3f25d9e9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:26.995188 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a506e9-c07c-49a1-9f8d-8eb3f25d9e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:26.995460 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a506e9-c07c-49a1-9f8d-8eb3f25d9e9b admin admin] PATH_INFO: `/v3/users/19457c759f0b4f5cbc530364af1f47d7` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:26.996837 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a506e9-c07c-49a1-9f8d-8eb3f25d9e9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=19457c759f0b4f5cbc530364af1f47d7)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:26.998839 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a506e9-c07c-49a1-9f8d-8eb3f25d9e9b admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:27.035189 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:27.080403 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-46a506e9-c07c-49a1-9f8d-8eb3f25d9e9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19457c759f0b4f5cbc530364af1f47d7'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:27.094897 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-46a506e9-c07c-49a1-9f8d-8eb3f25d9e9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19457c759f0b4f5cbc530364af1f47d7'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:27.104085 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-46a506e9-c07c-49a1-9f8d-8eb3f25d9e9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19457c759f0b4f5cbc530364af1f47d7'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:27.113396 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-46a506e9-c07c-49a1-9f8d-8eb3f25d9e9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19457c759f0b4f5cbc530364af1f47d7'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:27.114174 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-46a506e9-c07c-49a1-9f8d-8eb3f25d9e9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19457c759f0b4f5cbc530364af1f47d7'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:27.123284 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1185/2394] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:30:26 2022] DELETE /identity/v3/users/19457c759f0b4f5cbc530364af1f47d7 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:27.251753 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df75fae-a360-45cf-a356-2a0d837761ac service placement] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:27.253215 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-5df75fae-a360-45cf-a356-2a0d837761ac service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:27.268098 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df75fae-a360-45cf-a356-2a0d837761ac service placement] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:27.360254 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a506e9-c07c-49a1-9f8d-8eb3f25d9e9b admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:27.361801 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-46a506e9-c07c-49a1-9f8d-8eb3f25d9e9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:27.365569 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e300868e-7670-4c2e-9add-fcd54702065f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:27.366775 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300868e-7670-4c2e-9add-fcd54702065f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:27.367144 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300868e-7670-4c2e-9add-fcd54702065f admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:27.367509 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e300868e-7670-4c2e-9add-fcd54702065f admin admin] PATH_INFO: `/v3/projects/c92e403fe66644db94aa3905f992b420` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:27.369002 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e300868e-7670-4c2e-9add-fcd54702065f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c92e403fe66644db94aa3905f992b420)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:27.372730 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e300868e-7670-4c2e-9add-fcd54702065f admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:27.378530 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a506e9-c07c-49a1-9f8d-8eb3f25d9e9b admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:27.386573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595b1bf-52cd-4cfe-bce6-3f6046a2aafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:27.387799 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595b1bf-52cd-4cfe-bce6-3f6046a2aafd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:27.387937 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595b1bf-52cd-4cfe-bce6-3f6046a2aafd admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:27.388136 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595b1bf-52cd-4cfe-bce6-3f6046a2aafd admin admin] PATH_INFO: `/v3/projects/9351d2e0e0e14585820951a9398b35f4` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:27.389427 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a595b1bf-52cd-4cfe-bce6-3f6046a2aafd admin admin] RBAC: Authorizing `identity:delete_project(project_id=9351d2e0e0e14585820951a9398b35f4)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:27.392299 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a595b1bf-52cd-4cfe-bce6-3f6046a2aafd admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:27.461150 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-e300868e-7670-4c2e-9add-fcd54702065f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c92e403fe66644db94aa3905f992b420', 'request_id': 'req-e300868e-7670-4c2e-9add-fcd54702065f'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:27.466267 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-a595b1bf-52cd-4cfe-bce6-3f6046a2aafd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9351d2e0e0e14585820951a9398b35f4', 'request_id': 'req-a595b1bf-52cd-4cfe-bce6-3f6046a2aafd'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:27.467878 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-e300868e-7670-4c2e-9add-fcd54702065f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c92e403fe66644db94aa3905f992b420', 'request_id': 'req-e300868e-7670-4c2e-9add-fcd54702065f'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:27.474014 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-a595b1bf-52cd-4cfe-bce6-3f6046a2aafd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9351d2e0e0e14585820951a9398b35f4', 'request_id': 'req-a595b1bf-52cd-4cfe-bce6-3f6046a2aafd'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:27.480590 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-a595b1bf-52cd-4cfe-bce6-3f6046a2aafd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9351d2e0e0e14585820951a9398b35f4', 'request_id': 'req-a595b1bf-52cd-4cfe-bce6-3f6046a2aafd'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:27.481287 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-e300868e-7670-4c2e-9add-fcd54702065f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c92e403fe66644db94aa3905f992b420', 'request_id': 'req-e300868e-7670-4c2e-9add-fcd54702065f'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:27.500030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-a595b1bf-52cd-4cfe-bce6-3f6046a2aafd admin admin] The token cache is being invalidate because project 9351d2e0e0e14585820951a9398b35f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:30:27.500415 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-a595b1bf-52cd-4cfe-bce6-3f6046a2aafd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:27.502456 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-a595b1bf-52cd-4cfe-bce6-3f6046a2aafd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:27.504380 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1186/2395] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:30:27 2022] DELETE /identity/v3/projects/9351d2e0e0e14585820951a9398b35f4 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:27.505093 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-e300868e-7670-4c2e-9add-fcd54702065f admin admin] The token cache is being invalidate because project c92e403fe66644db94aa3905f992b420 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:30:27.505384 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-e300868e-7670-4c2e-9add-fcd54702065f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:27.506701 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-e300868e-7670-4c2e-9add-fcd54702065f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:27.511894 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1210/2396] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:30:27 2022] DELETE /identity/v3/projects/c92e403fe66644db94aa3905f992b420 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:27.512081 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f2373-c2e0-426e-985b-626ba5555577 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:27.512081 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f2373-c2e0-426e-985b-626ba5555577 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:27.512081 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f2373-c2e0-426e-985b-626ba5555577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:27.512894 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e300868e-7670-4c2e-9add-fcd54702065f admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:27.514598 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-e300868e-7670-4c2e-9add-fcd54702065f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:27.534367 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-951f2373-c2e0-426e-985b-626ba5555577 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:27.535938 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-951f2373-c2e0-426e-985b-626ba5555577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:27.597513 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e300868e-7670-4c2e-9add-fcd54702065f admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:27.608727 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1187/2397] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:27.616815 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e9904-2726-497a-b49c-79e6d9423596 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:27.617292 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e9904-2726-497a-b49c-79e6d9423596 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:27.617773 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889e9904-2726-497a-b49c-79e6d9423596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:27.637304 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:27.639018 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-889e9904-2726-497a-b49c-79e6d9423596 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:27.640272 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-889e9904-2726-497a-b49c-79e6d9423596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:27.648858 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1188/2398] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:30:27.653247 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e9904-2726-497a-b49c-79e6d9423596 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:27.662218 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889e9904-2726-497a-b49c-79e6d9423596 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:27.666843 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63da2e-cc3f-4e57-a733-9aef21512312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:27.667509 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48419184-39c3-49a6-a3e1-a0ebfad42143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:27.667630 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63da2e-cc3f-4e57-a733-9aef21512312 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:27.667770 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63da2e-cc3f-4e57-a733-9aef21512312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:27.668048 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63da2e-cc3f-4e57-a733-9aef21512312 admin admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:27.668508 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48419184-39c3-49a6-a3e1-a0ebfad42143 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:27.668783 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63da2e-cc3f-4e57-a733-9aef21512312 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:27.668877 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48419184-39c3-49a6-a3e1-a0ebfad42143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:27.669083 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48419184-39c3-49a6-a3e1-a0ebfad42143 admin admin] PATH_INFO: `/v3/users/3884f0125b604820b548db7564afb574` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:27.669194 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63da2e-cc3f-4e57-a733-9aef21512312 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:27.670257 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48419184-39c3-49a6-a3e1-a0ebfad42143 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3884f0125b604820b548db7564afb574)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:27.670721 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63da2e-cc3f-4e57-a733-9aef21512312 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:27.672167 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48419184-39c3-49a6-a3e1-a0ebfad42143 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:27.683066 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1189/2399] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:30:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:30:27.687452 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63da2e-cc3f-4e57-a733-9aef21512312 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:27.697326 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63da2e-cc3f-4e57-a733-9aef21512312 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:27.702111 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043453d-b2c3-44d5-8d3a-c0ca582132c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:27.702840 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043453d-b2c3-44d5-8d3a-c0ca582132c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:27.703081 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043453d-b2c3-44d5-8d3a-c0ca582132c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:27.703375 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043453d-b2c3-44d5-8d3a-c0ca582132c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:27.704093 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6043453d-b2c3-44d5-8d3a-c0ca582132c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:27.706084 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6043453d-b2c3-44d5-8d3a-c0ca582132c5 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:27.710650 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:27.723283 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-48419184-39c3-49a6-a3e1-a0ebfad42143 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3884f0125b604820b548db7564afb574'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:27.724057 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-48419184-39c3-49a6-a3e1-a0ebfad42143 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3884f0125b604820b548db7564afb574'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:27.733480 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-48419184-39c3-49a6-a3e1-a0ebfad42143 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3884f0125b604820b548db7564afb574'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:27.734202 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-48419184-39c3-49a6-a3e1-a0ebfad42143 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3884f0125b604820b548db7564afb574'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:27.739007 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1190/2400] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:30:27 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:27.743450 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043453d-b2c3-44d5-8d3a-c0ca582132c5 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:27.745202 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-6043453d-b2c3-44d5-8d3a-c0ca582132c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:27.751066 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-48419184-39c3-49a6-a3e1-a0ebfad42143 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3884f0125b604820b548db7564afb574'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:27.764768 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1211/2401] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:30:27 2022] DELETE /identity/v3/users/3884f0125b604820b548db7564afb574 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:27.816378 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043453d-b2c3-44d5-8d3a-c0ca582132c5 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:27.877875 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505343cb-d894-43d5-92cc-4c8a74019a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:27.878648 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505343cb-d894-43d5-92cc-4c8a74019a3b admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:27.879000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505343cb-d894-43d5-92cc-4c8a74019a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:27.879353 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505343cb-d894-43d5-92cc-4c8a74019a3b admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:27.880134 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505343cb-d894-43d5-92cc-4c8a74019a3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:27.882599 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505343cb-d894-43d5-92cc-4c8a74019a3b admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:27.905516 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1191/2402] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:30:27 2022] POST /identity/v3/users => generated 287 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:30:27.909912 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48419184-39c3-49a6-a3e1-a0ebfad42143 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:27.921936 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48419184-39c3-49a6-a3e1-a0ebfad42143 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:27.931787 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a91d52-df2d-4f2b-9b58-f760ec9aa2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:27.932654 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a91d52-df2d-4f2b-9b58-f760ec9aa2b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:27.933008 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a91d52-df2d-4f2b-9b58-f760ec9aa2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:27.933375 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a91d52-df2d-4f2b-9b58-f760ec9aa2b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:27.934233 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a91d52-df2d-4f2b-9b58-f760ec9aa2b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:27.936132 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a91d52-df2d-4f2b-9b58-f760ec9aa2b9 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:27.947664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1212/2403] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:30:27.952128 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505343cb-d894-43d5-92cc-4c8a74019a3b admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:27.962418 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505343cb-d894-43d5-92cc-4c8a74019a3b admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:27.967077 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f954e2e0-a5ad-41f0-9ab0-263206835a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:27.967751 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954e2e0-a5ad-41f0-9ab0-263206835a94 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:27.968035 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954e2e0-a5ad-41f0-9ab0-263206835a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:27.968312 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954e2e0-a5ad-41f0-9ab0-263206835a94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:27.969082 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f954e2e0-a5ad-41f0-9ab0-263206835a94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:27.970702 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f954e2e0-a5ad-41f0-9ab0-263206835a94 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:27.981610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1192/2404] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:30:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:27.985966 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a91d52-df2d-4f2b-9b58-f760ec9aa2b9 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:27.998168 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a91d52-df2d-4f2b-9b58-f760ec9aa2b9 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:28.005076 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0034a40-25a9-4e5c-a42a-6c592f1dc8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:28.006974 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0034a40-25a9-4e5c-a42a-6c592f1dc8a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:28.007453 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0034a40-25a9-4e5c-a42a-6c592f1dc8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:28.007986 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0034a40-25a9-4e5c-a42a-6c592f1dc8a3 admin admin] PATH_INFO: `/v3/projects/81f6783e23914aedb665d3f39c769f21/users/48b23be264b746eb9d629c3d403b34dd/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:28.035413 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0034a40-25a9-4e5c-a42a-6c592f1dc8a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81f6783e23914aedb665d3f39c769f21, user_id=48b23be264b746eb9d629c3d403b34dd, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:28.039053 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0034a40-25a9-4e5c-a42a-6c592f1dc8a3 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:28.057592 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1213/2405] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:30:27 2022] PUT /identity/v3/projects/81f6783e23914aedb665d3f39c769f21/users/48b23be264b746eb9d629c3d403b34dd/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:30:28.069477 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36985bbe-b1aa-418e-bd89-aad2a581745a None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:28.069898 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36985bbe-b1aa-418e-bd89-aad2a581745a None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:28.070769 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36985bbe-b1aa-418e-bd89-aad2a581745a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:28.071449 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0034a40-25a9-4e5c-a42a-6c592f1dc8a3 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:28.073376 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-c0034a40-25a9-4e5c-a42a-6c592f1dc8a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:28.101289 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-36985bbe-b1aa-418e-bd89-aad2a581745a None None] MFA Rules not processed for user `48b23be264b746eb9d629c3d403b34dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:28.102682 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-36985bbe-b1aa-418e-bd89-aad2a581745a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:28.155518 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0034a40-25a9-4e5c-a42a-6c592f1dc8a3 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:28.161096 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a8a69-3824-447e-8f1a-f161e093af6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:28.162242 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a8a69-3824-447e-8f1a-f161e093af6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:28.162520 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a8a69-3824-447e-8f1a-f161e093af6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:28.162792 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a8a69-3824-447e-8f1a-f161e093af6b admin admin] PATH_INFO: `/v3/projects/5300ce567c4341a9a8854577e2c7a29d` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:28.164089 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8a8a69-3824-447e-8f1a-f161e093af6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5300ce567c4341a9a8854577e2c7a29d)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:28.166067 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8a8a69-3824-447e-8f1a-f161e093af6b admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:28.177053 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:28.181118 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1193/2406] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:30:28.189863 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:28.192656 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36985bbe-b1aa-418e-bd89-aad2a581745a None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:28.194456 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-36985bbe-b1aa-418e-bd89-aad2a581745a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:28.219554 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-4d8a8a69-3824-447e-8f1a-f161e093af6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5300ce567c4341a9a8854577e2c7a29d', 'request_id': 'req-4d8a8a69-3824-447e-8f1a-f161e093af6b'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:28.226606 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-4d8a8a69-3824-447e-8f1a-f161e093af6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5300ce567c4341a9a8854577e2c7a29d', 'request_id': 'req-4d8a8a69-3824-447e-8f1a-f161e093af6b'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:28.235506 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-4d8a8a69-3824-447e-8f1a-f161e093af6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5300ce567c4341a9a8854577e2c7a29d', 'request_id': 'req-4d8a8a69-3824-447e-8f1a-f161e093af6b'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:28.246596 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-4d8a8a69-3824-447e-8f1a-f161e093af6b admin admin] The token cache is being invalidate because project 5300ce567c4341a9a8854577e2c7a29d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:30:28.246886 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-4d8a8a69-3824-447e-8f1a-f161e093af6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:28.247643 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-4d8a8a69-3824-447e-8f1a-f161e093af6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:28.253580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1214/2407] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:30:28 2022] DELETE /identity/v3/projects/5300ce567c4341a9a8854577e2c7a29d => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:30:28.255144 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37acdb1a-2b33-4cdf-9dc6-ce51f98fcb14 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:28.255515 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37acdb1a-2b33-4cdf-9dc6-ce51f98fcb14 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:28.255796 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37acdb1a-2b33-4cdf-9dc6-ce51f98fcb14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:28.274364 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-37acdb1a-2b33-4cdf-9dc6-ce51f98fcb14 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:28.275577 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-37acdb1a-2b33-4cdf-9dc6-ce51f98fcb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:28.277282 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36985bbe-b1aa-418e-bd89-aad2a581745a None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:28.341939 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0557e82d-4d1a-46ce-ae26-87afb115ce2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:28.342412 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1215/2408] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:28.345164 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0557e82d-4d1a-46ce-ae26-87afb115ce2c service nova] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:28.345473 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0557e82d-4d1a-46ce-ae26-87afb115ce2c service nova] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:28.345757 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0557e82d-4d1a-46ce-ae26-87afb115ce2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:28.347558 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-0557e82d-4d1a-46ce-ae26-87afb115ce2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:28.351017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d95b0-304e-455c-b633-50d7b6d78c41 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:28.351411 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d95b0-304e-455c-b633-50d7b6d78c41 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:28.351762 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d95b0-304e-455c-b633-50d7b6d78c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:28.372824 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-972d95b0-304e-455c-b633-50d7b6d78c41 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:28.374148 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-972d95b0-304e-455c-b633-50d7b6d78c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:28.383657 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1216/2409] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:28.387653 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972d95b0-304e-455c-b633-50d7b6d78c41 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:28.397154 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972d95b0-304e-455c-b633-50d7b6d78c41 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:28.401810 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c5062-d182-452e-b31f-a16d4f91f262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:28.402483 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c5062-d182-452e-b31f-a16d4f91f262 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:28.402766 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c5062-d182-452e-b31f-a16d4f91f262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:28.403046 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c5062-d182-452e-b31f-a16d4f91f262 admin admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:28.403770 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68c5062-d182-452e-b31f-a16d4f91f262 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:28.404076 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68c5062-d182-452e-b31f-a16d4f91f262 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:28.405789 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68c5062-d182-452e-b31f-a16d4f91f262 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:28.410085 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0557e82d-4d1a-46ce-ae26-87afb115ce2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:28.411796 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0557e82d-4d1a-46ce-ae26-87afb115ce2c service nova] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:28.421251 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1217/2410] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:30:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:30:28.422429 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1194/2411] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:30:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:30:28.426206 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c5062-d182-452e-b31f-a16d4f91f262 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:28.437585 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c5062-d182-452e-b31f-a16d4f91f262 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:28.442193 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e1023d-22fd-4ba5-940e-096f6abb75a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:28.442845 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1023d-22fd-4ba5-940e-096f6abb75a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:28.443137 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1023d-22fd-4ba5-940e-096f6abb75a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:28.443407 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1023d-22fd-4ba5-940e-096f6abb75a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:28.444079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e1023d-22fd-4ba5-940e-096f6abb75a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:28.447575 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e1023d-22fd-4ba5-940e-096f6abb75a9 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:28.485291 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1218/2412] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:30:28 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:28.486469 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0557e82d-4d1a-46ce-ae26-87afb115ce2c service nova] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:28.498114 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0557e82d-4d1a-46ce-ae26-87afb115ce2c service nova] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:28.536131 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:28.561168 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9055c9-db08-458a-8648-7664f63b9aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:28.562032 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9055c9-db08-458a-8648-7664f63b9aeb admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:28.562314 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9055c9-db08-458a-8648-7664f63b9aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:28.562586 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9055c9-db08-458a-8648-7664f63b9aeb admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:28.563282 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9055c9-db08-458a-8648-7664f63b9aeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:28.565210 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9055c9-db08-458a-8648-7664f63b9aeb admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:28.600612 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1195/2413] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:30:28 2022] POST /identity/v3/users => generated 287 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:30:28.602233 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e1023d-22fd-4ba5-940e-096f6abb75a9 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:28.617717 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e1023d-22fd-4ba5-940e-096f6abb75a9 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:28.622966 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e295ceba-13fd-4435-b984-e071a7b0c39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:28.623763 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295ceba-13fd-4435-b984-e071a7b0c39f admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:28.624341 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295ceba-13fd-4435-b984-e071a7b0c39f admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:28.624682 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e295ceba-13fd-4435-b984-e071a7b0c39f admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:28.625536 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e295ceba-13fd-4435-b984-e071a7b0c39f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:28.627275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e295ceba-13fd-4435-b984-e071a7b0c39f admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:28.641560 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1219/2414] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:28.642723 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9055c9-db08-458a-8648-7664f63b9aeb admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:28.656827 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9055c9-db08-458a-8648-7664f63b9aeb admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:28.662534 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad54e654-3063-4af9-93ca-35bdf6187a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:28.662534 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad54e654-3063-4af9-93ca-35bdf6187a2d admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:28.662534 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad54e654-3063-4af9-93ca-35bdf6187a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:28.662534 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad54e654-3063-4af9-93ca-35bdf6187a2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:28.662891 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad54e654-3063-4af9-93ca-35bdf6187a2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:28.664398 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad54e654-3063-4af9-93ca-35bdf6187a2d admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:28.667849 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e295ceba-13fd-4435-b984-e071a7b0c39f admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:28.669762 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-e295ceba-13fd-4435-b984-e071a7b0c39f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:28.684579 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1196/2415] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:28.687989 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e295ceba-13fd-4435-b984-e071a7b0c39f admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:28.688491 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad54e654-3063-4af9-93ca-35bdf6187a2d admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:28.692893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43853d43-32bf-4566-b8b9-eb63da40ad93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:28.694017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43853d43-32bf-4566-b8b9-eb63da40ad93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:28.694296 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43853d43-32bf-4566-b8b9-eb63da40ad93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:28.694569 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43853d43-32bf-4566-b8b9-eb63da40ad93 admin admin] PATH_INFO: `/v3/users/48b23be264b746eb9d629c3d403b34dd` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:28.696104 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43853d43-32bf-4566-b8b9-eb63da40ad93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48b23be264b746eb9d629c3d403b34dd)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:28.698155 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43853d43-32bf-4566-b8b9-eb63da40ad93 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:28.712501 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad54e654-3063-4af9-93ca-35bdf6187a2d admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:28.717210 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381aa532-a9c7-46ec-a093-1c66fca17bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:28.718107 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381aa532-a9c7-46ec-a093-1c66fca17bec admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:28.718387 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381aa532-a9c7-46ec-a093-1c66fca17bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:28.718721 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381aa532-a9c7-46ec-a093-1c66fca17bec admin admin] PATH_INFO: `/v3/projects/d144e28e9b8449bcb4ed2a3b1cc17788/users/48ab4cf593884190aec0973af11ea0f1/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:28.734813 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381aa532-a9c7-46ec-a093-1c66fca17bec admin admin] RBAC: Authorizing `identity:create_grant(project_id=d144e28e9b8449bcb4ed2a3b1cc17788, user_id=48ab4cf593884190aec0973af11ea0f1, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:28.737640 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381aa532-a9c7-46ec-a093-1c66fca17bec admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:28.752779 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1197/2416] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:30:28 2022] PUT /identity/v3/projects/d144e28e9b8449bcb4ed2a3b1cc17788/users/48ab4cf593884190aec0973af11ea0f1/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:30:28.761331 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-43853d43-32bf-4566-b8b9-eb63da40ad93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48b23be264b746eb9d629c3d403b34dd'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:28.761987 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-43853d43-32bf-4566-b8b9-eb63da40ad93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48b23be264b746eb9d629c3d403b34dd'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:28.765579 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:28.774313 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-43853d43-32bf-4566-b8b9-eb63da40ad93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48b23be264b746eb9d629c3d403b34dd'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:28.775331 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-43853d43-32bf-4566-b8b9-eb63da40ad93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48b23be264b746eb9d629c3d403b34dd'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:28.799732 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-43853d43-32bf-4566-b8b9-eb63da40ad93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48b23be264b746eb9d629c3d403b34dd'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:28.819939 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1220/2417] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:30:28 2022] DELETE /identity/v3/users/48b23be264b746eb9d629c3d403b34dd => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:30:28.822790 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de5860d-67b9-4542-b061-e70541263aca None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:28.823080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de5860d-67b9-4542-b061-e70541263aca None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:28.823355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de5860d-67b9-4542-b061-e70541263aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:28.826512 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2bdf2-e5fb-47a4-b03d-202d61f7a3a5 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:28.826794 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2bdf2-e5fb-47a4-b03d-202d61f7a3a5 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:28.827064 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2bdf2-e5fb-47a4-b03d-202d61f7a3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:28.843410 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-6de5860d-67b9-4542-b061-e70541263aca None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:28.844636 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-6de5860d-67b9-4542-b061-e70541263aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:28.856983 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-0ad2bdf2-e5fb-47a4-b03d-202d61f7a3a5 None None] MFA Rules not processed for user `48ab4cf593884190aec0973af11ea0f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:28.858392 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-0ad2bdf2-e5fb-47a4-b03d-202d61f7a3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:28.910177 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1198/2418] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:28.920289 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de5860d-67b9-4542-b061-e70541263aca None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:28.921782 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-6de5860d-67b9-4542-b061-e70541263aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:28.924512 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1221/2419] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:28.936362 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad2bdf2-e5fb-47a4-b03d-202d61f7a3a5 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:28.937810 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-0ad2bdf2-e5fb-47a4-b03d-202d61f7a3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:28.978677 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de5860d-67b9-4542-b061-e70541263aca None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:28.982499 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51218f3f-7cca-4ed1-8627-f00a399bfbbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:28.983147 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51218f3f-7cca-4ed1-8627-f00a399bfbbe service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:28.983428 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51218f3f-7cca-4ed1-8627-f00a399bfbbe service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:28.983702 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51218f3f-7cca-4ed1-8627-f00a399bfbbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:28.993315 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51218f3f-7cca-4ed1-8627-f00a399bfbbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:28.994920 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51218f3f-7cca-4ed1-8627-f00a399bfbbe service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:28.995427 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad2bdf2-e5fb-47a4-b03d-202d61f7a3a5 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:28.999394 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f686f391-c473-46cb-b567-119310854258 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:29.000042 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686f391-c473-46cb-b567-119310854258 service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:29.000340 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686f391-c473-46cb-b567-119310854258 service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:29.000579 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686f391-c473-46cb-b567-119310854258 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:29.005886 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1199/2420] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:30:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:30:29.010664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f686f391-c473-46cb-b567-119310854258 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:29.012128 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f686f391-c473-46cb-b567-119310854258 service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:29.022090 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1222/2421] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:30:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:30:29.042592 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51218f3f-7cca-4ed1-8627-f00a399bfbbe service neutron] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:29.044262 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-51218f3f-7cca-4ed1-8627-f00a399bfbbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:29.059472 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51218f3f-7cca-4ed1-8627-f00a399bfbbe service neutron] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:29.065244 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d63169-4b6a-4b5b-800e-51cee5ae6956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:29.066136 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d63169-4b6a-4b5b-800e-51cee5ae6956 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:29.066417 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d63169-4b6a-4b5b-800e-51cee5ae6956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:29.066733 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d63169-4b6a-4b5b-800e-51cee5ae6956 admin admin] PATH_INFO: `/v3/users/48ab4cf593884190aec0973af11ea0f1` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:29.068134 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d63169-4b6a-4b5b-800e-51cee5ae6956 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48ab4cf593884190aec0973af11ea0f1)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:29.070466 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d63169-4b6a-4b5b-800e-51cee5ae6956 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:29.103917 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:29.129834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-13d63169-4b6a-4b5b-800e-51cee5ae6956 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48ab4cf593884190aec0973af11ea0f1'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:29.142920 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-13d63169-4b6a-4b5b-800e-51cee5ae6956 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48ab4cf593884190aec0973af11ea0f1'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:29.154472 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-13d63169-4b6a-4b5b-800e-51cee5ae6956 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48ab4cf593884190aec0973af11ea0f1'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:29.164020 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-13d63169-4b6a-4b5b-800e-51cee5ae6956 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48ab4cf593884190aec0973af11ea0f1'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:29.164829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-13d63169-4b6a-4b5b-800e-51cee5ae6956 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48ab4cf593884190aec0973af11ea0f1'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:29.171843 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1200/2422] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:30:29 2022] DELETE /identity/v3/users/48ab4cf593884190aec0973af11ea0f1 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:29.176395 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b6acca-de74-4db1-b8e3-280b82c2511b None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:29.176759 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b6acca-de74-4db1-b8e3-280b82c2511b None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:29.177148 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b6acca-de74-4db1-b8e3-280b82c2511b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:29.195769 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-50b6acca-de74-4db1-b8e3-280b82c2511b None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:29.196982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-50b6acca-de74-4db1-b8e3-280b82c2511b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:29.254391 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:29.265919 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1223/2423] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:29.276788 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d63169-4b6a-4b5b-800e-51cee5ae6956 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:29.278378 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-13d63169-4b6a-4b5b-800e-51cee5ae6956 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:29.285021 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b6acca-de74-4db1-b8e3-280b82c2511b None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:29.287097 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-50b6acca-de74-4db1-b8e3-280b82c2511b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:29.301697 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b6acca-de74-4db1-b8e3-280b82c2511b None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:29.307248 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce0252-6889-448b-8193-80d4b10c5c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:29.308114 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce0252-6889-448b-8193-80d4b10c5c70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:29.308393 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce0252-6889-448b-8193-80d4b10c5c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:29.308667 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce0252-6889-448b-8193-80d4b10c5c70 admin admin] PATH_INFO: `/v3/projects/81f6783e23914aedb665d3f39c769f21` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:29.309885 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce0252-6889-448b-8193-80d4b10c5c70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81f6783e23914aedb665d3f39c769f21)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:29.311860 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce0252-6889-448b-8193-80d4b10c5c70 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:29.351735 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d63169-4b6a-4b5b-800e-51cee5ae6956 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:29.364068 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-5cce0252-6889-448b-8193-80d4b10c5c70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81f6783e23914aedb665d3f39c769f21', 'request_id': 'req-5cce0252-6889-448b-8193-80d4b10c5c70'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:29.370597 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-5cce0252-6889-448b-8193-80d4b10c5c70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81f6783e23914aedb665d3f39c769f21', 'request_id': 'req-5cce0252-6889-448b-8193-80d4b10c5c70'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:29.378974 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-5cce0252-6889-448b-8193-80d4b10c5c70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81f6783e23914aedb665d3f39c769f21', 'request_id': 'req-5cce0252-6889-448b-8193-80d4b10c5c70'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:29.390168 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-5cce0252-6889-448b-8193-80d4b10c5c70 admin admin] The token cache is being invalidate because project 81f6783e23914aedb665d3f39c769f21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:30:29.390471 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-5cce0252-6889-448b-8193-80d4b10c5c70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:29.391094 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-5cce0252-6889-448b-8193-80d4b10c5c70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:29.392669 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1224/2424] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:30:29 2022] DELETE /identity/v3/projects/81f6783e23914aedb665d3f39c769f21 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:30:29.398049 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa0175-09e9-49f4-9476-4941ee9c05b5 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:29.398339 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa0175-09e9-49f4-9476-4941ee9c05b5 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:29.398631 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa0175-09e9-49f4-9476-4941ee9c05b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:29.401432 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2099f900-e022-4ece-a86b-86111d610ba6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:29.402166 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099f900-e022-4ece-a86b-86111d610ba6 service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:29.402449 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099f900-e022-4ece-a86b-86111d610ba6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:29.402724 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099f900-e022-4ece-a86b-86111d610ba6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:29.404529 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-2099f900-e022-4ece-a86b-86111d610ba6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:29.417387 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-ccfa0175-09e9-49f4-9476-4941ee9c05b5 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:29.418496 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-ccfa0175-09e9-49f4-9476-4941ee9c05b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:29.469152 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2099f900-e022-4ece-a86b-86111d610ba6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:29.470610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2099f900-e022-4ece-a86b-86111d610ba6 service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:29.482296 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1201/2425] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:30:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:30:29.486169 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1225/2426] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:30:29.494580 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869ce11-fb3c-4567-81e4-7432cc62f064 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:29.494864 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869ce11-fb3c-4567-81e4-7432cc62f064 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:29.495231 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869ce11-fb3c-4567-81e4-7432cc62f064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:29.516467 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-3869ce11-fb3c-4567-81e4-7432cc62f064 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:29.517904 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-3869ce11-fb3c-4567-81e4-7432cc62f064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:29.529478 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1202/2427] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:29.534144 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa0175-09e9-49f4-9476-4941ee9c05b5 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:29.544577 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa0175-09e9-49f4-9476-4941ee9c05b5 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:29.550383 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53ec83-0c71-46d4-adf1-0f8611266d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:29.551164 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53ec83-0c71-46d4-adf1-0f8611266d49 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:29.551441 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53ec83-0c71-46d4-adf1-0f8611266d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:29.551679 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53ec83-0c71-46d4-adf1-0f8611266d49 admin admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:29.552841 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f53ec83-0c71-46d4-adf1-0f8611266d49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:29.553373 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f53ec83-0c71-46d4-adf1-0f8611266d49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:29.555070 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f53ec83-0c71-46d4-adf1-0f8611266d49 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:29.567667 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1226/2428] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:30:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:29.572514 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3869ce11-fb3c-4567-81e4-7432cc62f064 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:29.578255 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:29.582484 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3869ce11-fb3c-4567-81e4-7432cc62f064 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:29.588982 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917add7b-2ec2-4c5a-9c06-c77256d75bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:29.590904 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917add7b-2ec2-4c5a-9c06-c77256d75bdc admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:29.590904 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917add7b-2ec2-4c5a-9c06-c77256d75bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:29.591159 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917add7b-2ec2-4c5a-9c06-c77256d75bdc admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:29.592444 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917add7b-2ec2-4c5a-9c06-c77256d75bdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:29.594544 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917add7b-2ec2-4c5a-9c06-c77256d75bdc admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:29.625453 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1203/2429] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:30:29 2022] POST /identity/v3/projects => generated 363 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:29.629743 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53ec83-0c71-46d4-adf1-0f8611266d49 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:29.644579 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53ec83-0c71-46d4-adf1-0f8611266d49 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:29.704053 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dce59f9-eb9b-486c-b5be-5d4891b0a2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:29.704730 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce59f9-eb9b-486c-b5be-5d4891b0a2c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:29.705065 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce59f9-eb9b-486c-b5be-5d4891b0a2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:29.705343 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce59f9-eb9b-486c-b5be-5d4891b0a2c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:29.706041 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dce59f9-eb9b-486c-b5be-5d4891b0a2c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:29.708028 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dce59f9-eb9b-486c-b5be-5d4891b0a2c0 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:29.725468 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:29.731370 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1227/2430] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:30:29 2022] POST /identity/v3/users => generated 282 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:29.735664 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917add7b-2ec2-4c5a-9c06-c77256d75bdc admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:29.749990 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917add7b-2ec2-4c5a-9c06-c77256d75bdc admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:29.756904 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d0d70c-9b3b-420d-abb2-91057fcd640a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:29.757619 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0d70c-9b3b-420d-abb2-91057fcd640a admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:29.757898 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0d70c-9b3b-420d-abb2-91057fcd640a admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:29.758177 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0d70c-9b3b-420d-abb2-91057fcd640a admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:29.758914 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d0d70c-9b3b-420d-abb2-91057fcd640a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:29.760723 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d0d70c-9b3b-420d-abb2-91057fcd640a admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:29.769433 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dce59f9-eb9b-486c-b5be-5d4891b0a2c0 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:29.771703 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1204/2431] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:30:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:29.771890 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-5dce59f9-eb9b-486c-b5be-5d4891b0a2c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:29.775687 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d0d70c-9b3b-420d-abb2-91057fcd640a admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:29.785475 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dce59f9-eb9b-486c-b5be-5d4891b0a2c0 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:29.791306 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a73cc7-867a-4a27-8d9e-29dc2b82b7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:29.792773 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a73cc7-867a-4a27-8d9e-29dc2b82b7c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:29.793251 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a73cc7-867a-4a27-8d9e-29dc2b82b7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:29.793664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a73cc7-867a-4a27-8d9e-29dc2b82b7c4 admin admin] PATH_INFO: `/v3/projects/d144e28e9b8449bcb4ed2a3b1cc17788` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:29.795284 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d0d70c-9b3b-420d-abb2-91057fcd640a admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:29.795738 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a73cc7-867a-4a27-8d9e-29dc2b82b7c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d144e28e9b8449bcb4ed2a3b1cc17788)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:29.799440 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a73cc7-867a-4a27-8d9e-29dc2b82b7c4 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:29.801528 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf472fd7-0e8f-4aa2-bfa8-797ea716a5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:29.802313 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf472fd7-0e8f-4aa2-bfa8-797ea716a5cb admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:29.802703 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf472fd7-0e8f-4aa2-bfa8-797ea716a5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:29.802983 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf472fd7-0e8f-4aa2-bfa8-797ea716a5cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:29.803713 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf472fd7-0e8f-4aa2-bfa8-797ea716a5cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:29.805478 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf472fd7-0e8f-4aa2-bfa8-797ea716a5cb admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:29.818713 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1205/2432] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:30:29 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:30:29.823345 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf472fd7-0e8f-4aa2-bfa8-797ea716a5cb admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:29.833471 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf472fd7-0e8f-4aa2-bfa8-797ea716a5cb admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:29.839444 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46190ba8-1734-427b-83f4-e45dd5ddb187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:29.840344 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46190ba8-1734-427b-83f4-e45dd5ddb187 admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:29.840751 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46190ba8-1734-427b-83f4-e45dd5ddb187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:29.841013 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46190ba8-1734-427b-83f4-e45dd5ddb187 admin admin] PATH_INFO: `/v3/projects/35a4df3309f14eecb90bedb54519e1af/users/f555b65bfd174315be564fa3c3ee18cc/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:29.857117 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46190ba8-1734-427b-83f4-e45dd5ddb187 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35a4df3309f14eecb90bedb54519e1af, user_id=f555b65bfd174315be564fa3c3ee18cc, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:29.859793 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46190ba8-1734-427b-83f4-e45dd5ddb187 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:29.860458 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-61a73cc7-867a-4a27-8d9e-29dc2b82b7c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd144e28e9b8449bcb4ed2a3b1cc17788', 'request_id': 'req-61a73cc7-867a-4a27-8d9e-29dc2b82b7c4'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:29.868449 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-61a73cc7-867a-4a27-8d9e-29dc2b82b7c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd144e28e9b8449bcb4ed2a3b1cc17788', 'request_id': 'req-61a73cc7-867a-4a27-8d9e-29dc2b82b7c4'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:29.874259 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1206/2433] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:30:29 2022] PUT /identity/v3/projects/35a4df3309f14eecb90bedb54519e1af/users/f555b65bfd174315be564fa3c3ee18cc/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:30:29.878920 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-61a73cc7-867a-4a27-8d9e-29dc2b82b7c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd144e28e9b8449bcb4ed2a3b1cc17788', 'request_id': 'req-61a73cc7-867a-4a27-8d9e-29dc2b82b7c4'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:29.886447 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dbddb9-7d18-446b-acb6-4c30dce27504 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:29.886664 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dbddb9-7d18-446b-acb6-4c30dce27504 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:29.886975 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dbddb9-7d18-446b-acb6-4c30dce27504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:29.891827 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-61a73cc7-867a-4a27-8d9e-29dc2b82b7c4 admin admin] The token cache is being invalidate because project d144e28e9b8449bcb4ed2a3b1cc17788 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:30:29.892312 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-61a73cc7-867a-4a27-8d9e-29dc2b82b7c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:29.893158 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-61a73cc7-867a-4a27-8d9e-29dc2b82b7c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:29.896209 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1228/2434] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:30:29 2022] DELETE /identity/v3/projects/d144e28e9b8449bcb4ed2a3b1cc17788 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:30:29.902002 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a340f-813a-45ba-ac59-ae7f02423545 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:29.902293 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a340f-813a-45ba-ac59-ae7f02423545 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:29.902566 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a340f-813a-45ba-ac59-ae7f02423545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:29.919136 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-69dbddb9-7d18-446b-acb6-4c30dce27504 None None] MFA Rules not processed for user `f555b65bfd174315be564fa3c3ee18cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:29.920452 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-69dbddb9-7d18-446b-acb6-4c30dce27504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:29.923940 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-150a340f-813a-45ba-ac59-ae7f02423545 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:29.925463 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-150a340f-813a-45ba-ac59-ae7f02423545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:29.994911 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1207/2435] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:29 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:29.995305 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1229/2436] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:29.999354 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dbddb9-7d18-446b-acb6-4c30dce27504 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:30.000754 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-69dbddb9-7d18-446b-acb6-4c30dce27504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:30.002627 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f9887-1cdd-4a42-a83b-06a1bfbfc1fa None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:30.002912 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f9887-1cdd-4a42-a83b-06a1bfbfc1fa None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:30.003182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826f9887-1cdd-4a42-a83b-06a1bfbfc1fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:30.019009 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dbddb9-7d18-446b-acb6-4c30dce27504 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:30.024846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc519f3-6534-4bda-8acf-2ba0890eb688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:30.025633 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-826f9887-1cdd-4a42-a83b-06a1bfbfc1fa None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:30.025792 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc519f3-6534-4bda-8acf-2ba0890eb688 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:30.025792 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc519f3-6534-4bda-8acf-2ba0890eb688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:30.026073 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc519f3-6534-4bda-8acf-2ba0890eb688 admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:30.026540 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-826f9887-1cdd-4a42-a83b-06a1bfbfc1fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:30.026746 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc519f3-6534-4bda-8acf-2ba0890eb688 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:30.028647 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc519f3-6534-4bda-8acf-2ba0890eb688 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:30.039081 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1230/2437] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:30.039994 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f9887-1cdd-4a42-a83b-06a1bfbfc1fa None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:30.049579 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826f9887-1cdd-4a42-a83b-06a1bfbfc1fa None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:30.060995 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1208/2438] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:30:29 2022] POST /identity/v3/projects => generated 363 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:30:30.062870 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3d54e9-52f6-4ff2-9020-dd69b5befdc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:30.063121 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc519f3-6534-4bda-8acf-2ba0890eb688 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:30.063556 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d54e9-52f6-4ff2-9020-dd69b5befdc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:30.063838 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d54e9-52f6-4ff2-9020-dd69b5befdc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:30.064212 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d54e9-52f6-4ff2-9020-dd69b5befdc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:30.064939 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3d54e9-52f6-4ff2-9020-dd69b5befdc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:30.065357 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3d54e9-52f6-4ff2-9020-dd69b5befdc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:30.067339 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3d54e9-52f6-4ff2-9020-dd69b5befdc1 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:30.083649 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1231/2439] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:30:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:30.088550 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3d54e9-52f6-4ff2-9020-dd69b5befdc1 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:30.098790 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3d54e9-52f6-4ff2-9020-dd69b5befdc1 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:30.133927 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc519f3-6534-4bda-8acf-2ba0890eb688 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:30.139643 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c83505-7303-41d2-90f8-8a790486844a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:30.140319 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c83505-7303-41d2-90f8-8a790486844a admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:30.140633 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c83505-7303-41d2-90f8-8a790486844a admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:30.140875 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c83505-7303-41d2-90f8-8a790486844a admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:30.142020 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c83505-7303-41d2-90f8-8a790486844a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:30.143981 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c83505-7303-41d2-90f8-8a790486844a admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:30.152984 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:30.161074 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284eb888-bd51-40c7-b005-884bd31776b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:30.162013 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284eb888-bd51-40c7-b005-884bd31776b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:30.162506 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284eb888-bd51-40c7-b005-884bd31776b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:30.162840 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284eb888-bd51-40c7-b005-884bd31776b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:30.163524 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284eb888-bd51-40c7-b005-884bd31776b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:30.165538 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284eb888-bd51-40c7-b005-884bd31776b8 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:30.167489 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1209/2440] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:30:30 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:30:30.171325 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c83505-7303-41d2-90f8-8a790486844a admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:30.179297 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:30.187981 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c83505-7303-41d2-90f8-8a790486844a admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:30.196185 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce27396c-51b2-476e-bbb2-ffa56db40e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:30.197111 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce27396c-51b2-476e-bbb2-ffa56db40e23 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:30.197373 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce27396c-51b2-476e-bbb2-ffa56db40e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:30.198239 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce27396c-51b2-476e-bbb2-ffa56db40e23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:30.198717 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce27396c-51b2-476e-bbb2-ffa56db40e23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:30.200372 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce27396c-51b2-476e-bbb2-ffa56db40e23 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:30.215232 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1210/2441] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:30:30.216334 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce27396c-51b2-476e-bbb2-ffa56db40e23 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:30.220944 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1232/2442] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:30:30 2022] POST /identity/v3/projects => generated 371 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:30:30.221971 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284eb888-bd51-40c7-b005-884bd31776b8 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:30.226732 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce27396c-51b2-476e-bbb2-ffa56db40e23 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:30.288542 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f1611-b18a-43ed-8676-9c8cb6bec9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:30.289480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f1611-b18a-43ed-8676-9c8cb6bec9e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:30.289777 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f1611-b18a-43ed-8676-9c8cb6bec9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:30.290047 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f1611-b18a-43ed-8676-9c8cb6bec9e1 admin admin] PATH_INFO: `/v3/projects/6da08a22252946bf8b9796123a337053/users/d20c383a3a564f76b1dadd14c786cd4e/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:30.291743 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284eb888-bd51-40c7-b005-884bd31776b8 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:30.298808 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10971967-0d4d-4c5b-8162-e75a66582db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:30.299521 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10971967-0d4d-4c5b-8162-e75a66582db9 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:30.299800 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10971967-0d4d-4c5b-8162-e75a66582db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:30.300080 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10971967-0d4d-4c5b-8162-e75a66582db9 admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:30.300775 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10971967-0d4d-4c5b-8162-e75a66582db9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:30.302962 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10971967-0d4d-4c5b-8162-e75a66582db9 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:30.307815 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946f1611-b18a-43ed-8676-9c8cb6bec9e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6da08a22252946bf8b9796123a337053, user_id=d20c383a3a564f76b1dadd14c786cd4e, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:30.310608 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946f1611-b18a-43ed-8676-9c8cb6bec9e1 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:30.325545 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1211/2443] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:30:30 2022] PUT /identity/v3/projects/6da08a22252946bf8b9796123a337053/users/d20c383a3a564f76b1dadd14c786cd4e/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:30.331366 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1233/2444] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:30:30 2022] POST /identity/v3/users => generated 286 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:30.331521 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f1611-b18a-43ed-8676-9c8cb6bec9e1 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:30.332346 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10971967-0d4d-4c5b-8162-e75a66582db9 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:30.342288 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10971967-0d4d-4c5b-8162-e75a66582db9 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:30.392446 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f1611-b18a-43ed-8676-9c8cb6bec9e1 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:30.397226 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3875792-8143-4adf-bb98-191684bd75ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:30.397900 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3875792-8143-4adf-bb98-191684bd75ef admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:30.398181 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3875792-8143-4adf-bb98-191684bd75ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:30.398586 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3875792-8143-4adf-bb98-191684bd75ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:30.398954 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a45622e-a49d-4ce8-9c22-9362ee50808e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:30.399294 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3875792-8143-4adf-bb98-191684bd75ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:30.399653 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45622e-a49d-4ce8-9c22-9362ee50808e admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:30.399933 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45622e-a49d-4ce8-9c22-9362ee50808e admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:30.400198 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45622e-a49d-4ce8-9c22-9362ee50808e admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:30.400944 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a45622e-a49d-4ce8-9c22-9362ee50808e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:30.401142 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3875792-8143-4adf-bb98-191684bd75ef admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:30.402835 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a45622e-a49d-4ce8-9c22-9362ee50808e admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:30.412062 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1212/2445] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:30:30.413349 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1234/2446] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:30:30.417028 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a45622e-a49d-4ce8-9c22-9362ee50808e admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:30.417153 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3875792-8143-4adf-bb98-191684bd75ef admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:30.426854 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a45622e-a49d-4ce8-9c22-9362ee50808e admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:30.430533 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3875792-8143-4adf-bb98-191684bd75ef admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:30.431624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6652167c-cb9c-4084-b825-9753d808d4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:30.432282 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6652167c-cb9c-4084-b825-9753d808d4f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:30.432564 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6652167c-cb9c-4084-b825-9753d808d4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:30.432837 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6652167c-cb9c-4084-b825-9753d808d4f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:30.433585 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6652167c-cb9c-4084-b825-9753d808d4f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:30.435202 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6652167c-cb9c-4084-b825-9753d808d4f1 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:30.435408 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee36050d-afa4-46e9-b6e1-d8adf4a43581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:30.436298 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36050d-afa4-46e9-b6e1-d8adf4a43581 admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:30.436575 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36050d-afa4-46e9-b6e1-d8adf4a43581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:30.436846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee36050d-afa4-46e9-b6e1-d8adf4a43581 admin admin] PATH_INFO: `/v3/projects/6da08a22252946bf8b9796123a337053/users/d20c383a3a564f76b1dadd14c786cd4e/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:30.438843 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee36050d-afa4-46e9-b6e1-d8adf4a43581 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6da08a22252946bf8b9796123a337053, user_id=d20c383a3a564f76b1dadd14c786cd4e, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:30.441518 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee36050d-afa4-46e9-b6e1-d8adf4a43581 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:30.445533 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1235/2447] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:30:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:30.449670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6652167c-cb9c-4084-b825-9753d808d4f1 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:30.453665 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1213/2448] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:30:30 2022] PUT /identity/v3/projects/6da08a22252946bf8b9796123a337053/users/d20c383a3a564f76b1dadd14c786cd4e/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:30.460808 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6652167c-cb9c-4084-b825-9753d808d4f1 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:30.465237 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c402a84-1e70-4a27-999f-6a64a6dc82d8 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:30.465526 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c402a84-1e70-4a27-999f-6a64a6dc82d8 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:30.465799 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c402a84-1e70-4a27-999f-6a64a6dc82d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:30.491929 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-4c402a84-1e70-4a27-999f-6a64a6dc82d8 None None] MFA Rules not processed for user `d20c383a3a564f76b1dadd14c786cd4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:30.493161 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-4c402a84-1e70-4a27-999f-6a64a6dc82d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:30.520422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f306539-98fc-4258-b806-ce2913facbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:30.522026 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f306539-98fc-4258-b806-ce2913facbf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:30.522308 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f306539-98fc-4258-b806-ce2913facbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:30.522584 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f306539-98fc-4258-b806-ce2913facbf2 admin admin] PATH_INFO: `/v3/projects/9966ab8ffdcf4e828553261c05fbb5a9/users/aa92e046c3274a29a8aa8857d7570750/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:30.531654 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:30.537052 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:30.542760 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f306539-98fc-4258-b806-ce2913facbf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9966ab8ffdcf4e828553261c05fbb5a9, user_id=aa92e046c3274a29a8aa8857d7570750, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:30.545822 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f306539-98fc-4258-b806-ce2913facbf2 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:30.562753 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1236/2449] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:30:30 2022] PUT /identity/v3/projects/9966ab8ffdcf4e828553261c05fbb5a9/users/aa92e046c3274a29a8aa8857d7570750/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:30:30.568160 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0937e-f155-483f-b644-98716bd4e138 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:30.568568 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0937e-f155-483f-b644-98716bd4e138 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:30.568903 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0937e-f155-483f-b644-98716bd4e138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:30.577942 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1214/2450] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:30 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:30:30.586478 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-59e0937e-f155-483f-b644-98716bd4e138 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:30.587822 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-59e0937e-f155-483f-b644-98716bd4e138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:30.591573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c402a84-1e70-4a27-999f-6a64a6dc82d8 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:30.593895 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-4c402a84-1e70-4a27-999f-6a64a6dc82d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:30.651152 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1237/2451] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:30:30.653949 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c402a84-1e70-4a27-999f-6a64a6dc82d8 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:30.661340 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e0937e-f155-483f-b644-98716bd4e138 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:30.663540 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-59e0937e-f155-483f-b644-98716bd4e138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:30.703423 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b584c9-c52a-46e0-aeb5-d8bd65683680 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:30.704079 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b584c9-c52a-46e0-aeb5-d8bd65683680 service cinder] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:30.704360 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b584c9-c52a-46e0-aeb5-d8bd65683680 service cinder] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:30.704627 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b584c9-c52a-46e0-aeb5-d8bd65683680 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:30.722368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e0937e-f155-483f-b644-98716bd4e138 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:30.727330 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d1dbf7-1295-41d9-8d22-e74ecbd28c54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:30.728145 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1dbf7-1295-41d9-8d22-e74ecbd28c54 service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:30.728487 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1dbf7-1295-41d9-8d22-e74ecbd28c54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:30.728766 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d1dbf7-1295-41d9-8d22-e74ecbd28c54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:30.738541 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d1dbf7-1295-41d9-8d22-e74ecbd28c54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:30.740070 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d1dbf7-1295-41d9-8d22-e74ecbd28c54 service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:30.749878 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1238/2452] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:30:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:30:30.767842 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b584c9-c52a-46e0-aeb5-d8bd65683680 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:30.769415 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b584c9-c52a-46e0-aeb5-d8bd65683680 service cinder] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:30.780652 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1215/2453] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:30:30 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:30:31.252996 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d1dbf7-1295-41d9-8d22-e74ecbd28c54 service neutron] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:31.265876 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d1dbf7-1295-41d9-8d22-e74ecbd28c54 service neutron] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:31.271133 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be3d4e-9694-4524-ad0b-928ffb7826d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:31.272017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be3d4e-9694-4524-ad0b-928ffb7826d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:31.272293 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be3d4e-9694-4524-ad0b-928ffb7826d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:31.272561 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be3d4e-9694-4524-ad0b-928ffb7826d2 admin admin] PATH_INFO: `/v3/users/f555b65bfd174315be564fa3c3ee18cc` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:31.273776 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52be3d4e-9694-4524-ad0b-928ffb7826d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f555b65bfd174315be564fa3c3ee18cc)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:31.276014 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52be3d4e-9694-4524-ad0b-928ffb7826d2 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:31.330375 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-52be3d4e-9694-4524-ad0b-928ffb7826d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f555b65bfd174315be564fa3c3ee18cc'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:31.331021 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-52be3d4e-9694-4524-ad0b-928ffb7826d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f555b65bfd174315be564fa3c3ee18cc'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:31.338943 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-52be3d4e-9694-4524-ad0b-928ffb7826d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f555b65bfd174315be564fa3c3ee18cc'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:31.339645 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-52be3d4e-9694-4524-ad0b-928ffb7826d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f555b65bfd174315be564fa3c3ee18cc'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:31.352219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-52be3d4e-9694-4524-ad0b-928ffb7826d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f555b65bfd174315be564fa3c3ee18cc'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:31.365416 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1239/2454] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:30:31 2022] DELETE /identity/v3/users/f555b65bfd174315be564fa3c3ee18cc => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:31.370574 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd98703-fb13-4929-99b9-7efe1d1bf9d2 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:31.370862 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd98703-fb13-4929-99b9-7efe1d1bf9d2 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:31.371128 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd98703-fb13-4929-99b9-7efe1d1bf9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:31.391841 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-7cd98703-fb13-4929-99b9-7efe1d1bf9d2 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:31.391841 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-7cd98703-fb13-4929-99b9-7efe1d1bf9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:31.427058 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:31.454851 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1216/2455] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:31.464892 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be3d4e-9694-4524-ad0b-928ffb7826d2 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:31.466349 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-52be3d4e-9694-4524-ad0b-928ffb7826d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:31.474250 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:31.540860 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be3d4e-9694-4524-ad0b-928ffb7826d2 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:31.544807 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c258c-7842-4449-bf51-da26b8e55782 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:31.545295 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c258c-7842-4449-bf51-da26b8e55782 service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:31.545573 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c258c-7842-4449-bf51-da26b8e55782 service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:31.545848 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c258c-7842-4449-bf51-da26b8e55782 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:31.551906 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e17908-c4b5-4494-8e02-e689ed11928b None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:31.552333 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e17908-c4b5-4494-8e02-e689ed11928b None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:31.552540 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e17908-c4b5-4494-8e02-e689ed11928b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:31.556036 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540c258c-7842-4449-bf51-da26b8e55782 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:31.557550 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540c258c-7842-4449-bf51-da26b8e55782 service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:31.567483 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1240/2456] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:30:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:30:31.584287 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-57e17908-c4b5-4494-8e02-e689ed11928b None None] MFA Rules not processed for user `aa92e046c3274a29a8aa8857d7570750`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:31.585552 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-57e17908-c4b5-4494-8e02-e689ed11928b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:31.700540 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1217/2457] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:31.713080 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c258c-7842-4449-bf51-da26b8e55782 service neutron] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:31.715100 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-540c258c-7842-4449-bf51-da26b8e55782 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:31.782278 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c258c-7842-4449-bf51-da26b8e55782 service neutron] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:31.785903 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445a14b-e4ec-4cd4-a928-3f4b9e3790ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:31.786587 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445a14b-e4ec-4cd4-a928-3f4b9e3790ed service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:31.786936 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445a14b-e4ec-4cd4-a928-3f4b9e3790ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:31.787428 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445a14b-e4ec-4cd4-a928-3f4b9e3790ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:31.797797 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445a14b-e4ec-4cd4-a928-3f4b9e3790ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:31.799260 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8445a14b-e4ec-4cd4-a928-3f4b9e3790ed service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:31.808733 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1241/2458] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:30:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:30:31.877812 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e17908-c4b5-4494-8e02-e689ed11928b None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:31.879324 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-57e17908-c4b5-4494-8e02-e689ed11928b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:31.894019 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e17908-c4b5-4494-8e02-e689ed11928b None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:31.903192 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286f75b-00b4-44b8-ac1c-e209d8d352c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:31.904096 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286f75b-00b4-44b8-ac1c-e209d8d352c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:31.904466 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286f75b-00b4-44b8-ac1c-e209d8d352c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:31.904672 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286f75b-00b4-44b8-ac1c-e209d8d352c3 admin admin] PATH_INFO: `/v3/projects/35a4df3309f14eecb90bedb54519e1af` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:31.905939 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4286f75b-00b4-44b8-ac1c-e209d8d352c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35a4df3309f14eecb90bedb54519e1af)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:31.907903 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4286f75b-00b4-44b8-ac1c-e209d8d352c3 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:31.957723 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-4286f75b-00b4-44b8-ac1c-e209d8d352c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35a4df3309f14eecb90bedb54519e1af', 'request_id': 'req-4286f75b-00b4-44b8-ac1c-e209d8d352c3'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:31.964293 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-4286f75b-00b4-44b8-ac1c-e209d8d352c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35a4df3309f14eecb90bedb54519e1af', 'request_id': 'req-4286f75b-00b4-44b8-ac1c-e209d8d352c3'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:31.970077 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-4286f75b-00b4-44b8-ac1c-e209d8d352c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35a4df3309f14eecb90bedb54519e1af', 'request_id': 'req-4286f75b-00b4-44b8-ac1c-e209d8d352c3'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:31.984407 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-4286f75b-00b4-44b8-ac1c-e209d8d352c3 admin admin] The token cache is being invalidate because project 35a4df3309f14eecb90bedb54519e1af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:30:31.984657 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-4286f75b-00b4-44b8-ac1c-e209d8d352c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:31.985297 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-4286f75b-00b4-44b8-ac1c-e209d8d352c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:31.986876 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1218/2459] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:30:31 2022] DELETE /identity/v3/projects/35a4df3309f14eecb90bedb54519e1af => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:31.991152 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445a14b-e4ec-4cd4-a928-3f4b9e3790ed service cinder] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:32.002916 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8445a14b-e4ec-4cd4-a928-3f4b9e3790ed service cinder] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:32.077009 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27994fe9-0450-45d6-87b8-0bf3a34efdd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:32.077009 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27994fe9-0450-45d6-87b8-0bf3a34efdd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:32.077009 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27994fe9-0450-45d6-87b8-0bf3a34efdd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:32.077527 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27994fe9-0450-45d6-87b8-0bf3a34efdd2 admin admin] PATH_INFO: `/v3/users/d20c383a3a564f76b1dadd14c786cd4e` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:32.078436 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27994fe9-0450-45d6-87b8-0bf3a34efdd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d20c383a3a564f76b1dadd14c786cd4e)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:32.080692 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27994fe9-0450-45d6-87b8-0bf3a34efdd2 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:32.147337 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-27994fe9-0450-45d6-87b8-0bf3a34efdd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd20c383a3a564f76b1dadd14c786cd4e'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:32.148026 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-27994fe9-0450-45d6-87b8-0bf3a34efdd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd20c383a3a564f76b1dadd14c786cd4e'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:32.150941 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:32.159911 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-27994fe9-0450-45d6-87b8-0bf3a34efdd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd20c383a3a564f76b1dadd14c786cd4e'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:32.161557 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-27994fe9-0450-45d6-87b8-0bf3a34efdd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd20c383a3a564f76b1dadd14c786cd4e'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:32.175829 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-27994fe9-0450-45d6-87b8-0bf3a34efdd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd20c383a3a564f76b1dadd14c786cd4e'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:32.188747 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1242/2460] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:30:31 2022] DELETE /identity/v3/users/d20c383a3a564f76b1dadd14c786cd4e => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:32.501889 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286f75b-00b4-44b8-ac1c-e209d8d352c3 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:32.503915 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-4286f75b-00b4-44b8-ac1c-e209d8d352c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:32.518279 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286f75b-00b4-44b8-ac1c-e209d8d352c3 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:32.576395 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:32.586371 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d62aec-2d76-4674-b09b-9e985bfb7c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:32.587273 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d62aec-2d76-4674-b09b-9e985bfb7c18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:32.587630 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d62aec-2d76-4674-b09b-9e985bfb7c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:32.587972 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d62aec-2d76-4674-b09b-9e985bfb7c18 admin admin] PATH_INFO: `/v3/projects/6da08a22252946bf8b9796123a337053` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:32.589268 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d62aec-2d76-4674-b09b-9e985bfb7c18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6da08a22252946bf8b9796123a337053)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:32.591180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d62aec-2d76-4674-b09b-9e985bfb7c18 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:32.639956 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-52d62aec-2d76-4674-b09b-9e985bfb7c18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6da08a22252946bf8b9796123a337053', 'request_id': 'req-52d62aec-2d76-4674-b09b-9e985bfb7c18'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:32.646268 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-52d62aec-2d76-4674-b09b-9e985bfb7c18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6da08a22252946bf8b9796123a337053', 'request_id': 'req-52d62aec-2d76-4674-b09b-9e985bfb7c18'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:32.651863 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-52d62aec-2d76-4674-b09b-9e985bfb7c18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6da08a22252946bf8b9796123a337053', 'request_id': 'req-52d62aec-2d76-4674-b09b-9e985bfb7c18'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:32.666677 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-52d62aec-2d76-4674-b09b-9e985bfb7c18 admin admin] The token cache is being invalidate because project 6da08a22252946bf8b9796123a337053 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:30:32.667811 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-52d62aec-2d76-4674-b09b-9e985bfb7c18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:32.669487 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-52d62aec-2d76-4674-b09b-9e985bfb7c18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:32.671130 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1219/2461] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:30:32 2022] DELETE /identity/v3/projects/6da08a22252946bf8b9796123a337053 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:32.676150 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe16c6d-06ca-47e3-993f-dded3f013246 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:32.676641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe16c6d-06ca-47e3-993f-dded3f013246 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:32.676836 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe16c6d-06ca-47e3-993f-dded3f013246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:32.696149 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-dbe16c6d-06ca-47e3-993f-dded3f013246 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:32.697426 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-dbe16c6d-06ca-47e3-993f-dded3f013246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:32.761098 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1243/2462] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:32.769197 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343bf52-b783-4ed6-972d-ce279ad9a2f9 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:32.769487 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343bf52-b783-4ed6-972d-ce279ad9a2f9 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:32.769764 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343bf52-b783-4ed6-972d-ce279ad9a2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:32.787931 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-8343bf52-b783-4ed6-972d-ce279ad9a2f9 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:32.789159 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-8343bf52-b783-4ed6-972d-ce279ad9a2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:32.797580 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1220/2463] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:32.801764 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe16c6d-06ca-47e3-993f-dded3f013246 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:32.812046 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe16c6d-06ca-47e3-993f-dded3f013246 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:32.818196 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fad7129-dec3-4a05-ae17-7562730c4616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:32.818846 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad7129-dec3-4a05-ae17-7562730c4616 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:32.819119 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad7129-dec3-4a05-ae17-7562730c4616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:32.819390 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad7129-dec3-4a05-ae17-7562730c4616 admin admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:32.820111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fad7129-dec3-4a05-ae17-7562730c4616 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:32.820408 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fad7129-dec3-4a05-ae17-7562730c4616 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:32.822082 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fad7129-dec3-4a05-ae17-7562730c4616 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:32.834586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1244/2464] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:30:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:30:32.838920 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8343bf52-b783-4ed6-972d-ce279ad9a2f9 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:32.848208 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8343bf52-b783-4ed6-972d-ce279ad9a2f9 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:32.853064 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32b954-5882-4e8c-a6a7-326f2ea28719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:32.853711 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32b954-5882-4e8c-a6a7-326f2ea28719 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:32.853992 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32b954-5882-4e8c-a6a7-326f2ea28719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:32.854261 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32b954-5882-4e8c-a6a7-326f2ea28719 admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:32.854954 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc32b954-5882-4e8c-a6a7-326f2ea28719 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:32.856857 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc32b954-5882-4e8c-a6a7-326f2ea28719 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:32.886061 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1221/2465] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:30:32 2022] POST /identity/v3/projects => generated 355 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:32.892038 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fad7129-dec3-4a05-ae17-7562730c4616 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:32.908572 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fad7129-dec3-4a05-ae17-7562730c4616 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:32.982257 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e715473-1b76-452f-83b3-b0d7da45e140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:32.982919 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e715473-1b76-452f-83b3-b0d7da45e140 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:32.983193 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e715473-1b76-452f-83b3-b0d7da45e140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:32.983462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e715473-1b76-452f-83b3-b0d7da45e140 admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:32.984145 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e715473-1b76-452f-83b3-b0d7da45e140 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:32.986119 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e715473-1b76-452f-83b3-b0d7da45e140 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:33.009570 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1245/2466] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:30:32 2022] POST /identity/v3/users => generated 278 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:33.013855 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32b954-5882-4e8c-a6a7-326f2ea28719 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:33.024725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32b954-5882-4e8c-a6a7-326f2ea28719 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:33.030213 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b80ca-5274-4c79-b68e-e5e8c21f3ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:33.030885 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b80ca-5274-4c79-b68e-e5e8c21f3ecc admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:33.031164 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b80ca-5274-4c79-b68e-e5e8c21f3ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:33.031452 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b80ca-5274-4c79-b68e-e5e8c21f3ecc admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:33.032196 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359b80ca-5274-4c79-b68e-e5e8c21f3ecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:33.033839 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359b80ca-5274-4c79-b68e-e5e8c21f3ecc admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:33.044935 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1222/2467] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:33.049142 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e715473-1b76-452f-83b3-b0d7da45e140 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:33.060494 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e715473-1b76-452f-83b3-b0d7da45e140 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:33.065908 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385df789-ea13-4c14-ba7f-273231c9957f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:33.066576 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385df789-ea13-4c14-ba7f-273231c9957f admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:33.066859 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385df789-ea13-4c14-ba7f-273231c9957f admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:33.067136 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385df789-ea13-4c14-ba7f-273231c9957f admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:33.067910 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385df789-ea13-4c14-ba7f-273231c9957f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:33.069557 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385df789-ea13-4c14-ba7f-273231c9957f admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:33.082391 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1246/2468] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:33.087734 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b80ca-5274-4c79-b68e-e5e8c21f3ecc admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:33.098586 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359b80ca-5274-4c79-b68e-e5e8c21f3ecc admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:33.104921 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea50b8-185d-466a-ad0e-222a80f15d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:33.105864 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea50b8-185d-466a-ad0e-222a80f15d59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:33.106142 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea50b8-185d-466a-ad0e-222a80f15d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:33.106412 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea50b8-185d-466a-ad0e-222a80f15d59 admin admin] PATH_INFO: `/v3/projects/226d9eeec7ef4e98b2b408b17deaf8af/users/4a4c1aca3ff2459d8c9585c163283332/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:33.123823 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea50b8-185d-466a-ad0e-222a80f15d59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=226d9eeec7ef4e98b2b408b17deaf8af, user_id=4a4c1aca3ff2459d8c9585c163283332, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:33.127180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea50b8-185d-466a-ad0e-222a80f15d59 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:33.144741 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1223/2469] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:30:33 2022] PUT /identity/v3/projects/226d9eeec7ef4e98b2b408b17deaf8af/users/4a4c1aca3ff2459d8c9585c163283332/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:33.156999 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0aada7-c68a-4d15-b2b4-cd09c5b3dfec None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:33.157295 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0aada7-c68a-4d15-b2b4-cd09c5b3dfec None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:33.157574 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0aada7-c68a-4d15-b2b4-cd09c5b3dfec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:33.187543 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-5b0aada7-c68a-4d15-b2b4-cd09c5b3dfec None None] MFA Rules not processed for user `4a4c1aca3ff2459d8c9585c163283332`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:33.188748 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-5b0aada7-c68a-4d15-b2b4-cd09c5b3dfec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:33.201423 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb9b93-326d-41e3-b1b4-6f532011ede1 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:33.201713 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb9b93-326d-41e3-b1b4-6f532011ede1 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:33.201994 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb9b93-326d-41e3-b1b4-6f532011ede1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:33.211204 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:33.220007 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-f9fb9b93-326d-41e3-b1b4-6f532011ede1 None None] MFA Rules not processed for user `aa92e046c3274a29a8aa8857d7570750`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:33.221270 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-f9fb9b93-326d-41e3-b1b4-6f532011ede1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:33.258845 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1247/2470] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:33 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:33.263956 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0aada7-c68a-4d15-b2b4-cd09c5b3dfec None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:33.281731 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1224/2471] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:30:33.336161 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0aada7-c68a-4d15-b2b4-cd09c5b3dfec None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:33.340728 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29346a30-0e05-4bf5-a0fd-ac8363e3be2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:33.341419 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29346a30-0e05-4bf5-a0fd-ac8363e3be2c admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:33.341697 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29346a30-0e05-4bf5-a0fd-ac8363e3be2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:33.341964 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29346a30-0e05-4bf5-a0fd-ac8363e3be2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:33.342751 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29346a30-0e05-4bf5-a0fd-ac8363e3be2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:33.344664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29346a30-0e05-4bf5-a0fd-ac8363e3be2c admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:33.372637 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1248/2472] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:30:33 2022] POST /identity/v3/projects => generated 357 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:30:33.376519 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb9b93-326d-41e3-b1b4-6f532011ede1 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:33.387964 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb9b93-326d-41e3-b1b4-6f532011ede1 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:33.413661 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:33.458383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25185ce3-8a0f-4f80-9237-cffe65fa7d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:33.459174 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25185ce3-8a0f-4f80-9237-cffe65fa7d90 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:33.459567 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25185ce3-8a0f-4f80-9237-cffe65fa7d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:33.459824 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25185ce3-8a0f-4f80-9237-cffe65fa7d90 admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:33.460518 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25185ce3-8a0f-4f80-9237-cffe65fa7d90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:33.462469 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25185ce3-8a0f-4f80-9237-cffe65fa7d90 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:33.484857 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1225/2473] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:30:33 2022] POST /identity/v3/users => generated 285 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:33.488864 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29346a30-0e05-4bf5-a0fd-ac8363e3be2c admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:33.503168 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29346a30-0e05-4bf5-a0fd-ac8363e3be2c admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:33.508860 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a68f1-8512-4a92-9e4a-68776c8763c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:33.509644 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a68f1-8512-4a92-9e4a-68776c8763c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:33.509925 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a68f1-8512-4a92-9e4a-68776c8763c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:33.510231 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a68f1-8512-4a92-9e4a-68776c8763c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:33.511531 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1a68f1-8512-4a92-9e4a-68776c8763c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:33.514335 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1a68f1-8512-4a92-9e4a-68776c8763c2 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:33.526172 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1249/2474] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:33.530747 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25185ce3-8a0f-4f80-9237-cffe65fa7d90 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:33.542344 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25185ce3-8a0f-4f80-9237-cffe65fa7d90 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:33.548060 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e94dc-7f5e-49cf-b00b-6f7223f20bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:33.549065 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e94dc-7f5e-49cf-b00b-6f7223f20bc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:33.549329 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e94dc-7f5e-49cf-b00b-6f7223f20bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:33.549606 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e94dc-7f5e-49cf-b00b-6f7223f20bc4 admin admin] PATH_INFO: `/v3/projects/a09b67b513264ee9afea8afddd11a594/users/7ac0a11e964e4132b5d9c7420e661225/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:33.568765 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e94dc-7f5e-49cf-b00b-6f7223f20bc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a09b67b513264ee9afea8afddd11a594, user_id=7ac0a11e964e4132b5d9c7420e661225, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:33.571734 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e94dc-7f5e-49cf-b00b-6f7223f20bc4 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:33.586817 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1226/2475] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:30:33 2022] PUT /identity/v3/projects/a09b67b513264ee9afea8afddd11a594/users/7ac0a11e964e4132b5d9c7420e661225/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:33.590903 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a68f1-8512-4a92-9e4a-68776c8763c2 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:33.603234 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a68f1-8512-4a92-9e4a-68776c8763c2 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:33.695176 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60245128-9d23-4b08-a5d8-e3fe36369d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:33.696045 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60245128-9d23-4b08-a5d8-e3fe36369d6d admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:33.696244 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60245128-9d23-4b08-a5d8-e3fe36369d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:33.696523 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60245128-9d23-4b08-a5d8-e3fe36369d6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:33.697375 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60245128-9d23-4b08-a5d8-e3fe36369d6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:33.700342 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60245128-9d23-4b08-a5d8-e3fe36369d6d admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:33.724311 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1250/2476] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:30:33 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:33.727837 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e94dc-7f5e-49cf-b00b-6f7223f20bc4 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:33.744815 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e94dc-7f5e-49cf-b00b-6f7223f20bc4 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:33.758343 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc5b50-381d-4a04-b573-c62b21736286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:33.759363 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc5b50-381d-4a04-b573-c62b21736286 admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:33.759774 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc5b50-381d-4a04-b573-c62b21736286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:33.759965 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc5b50-381d-4a04-b573-c62b21736286 admin admin] PATH_INFO: `/v3/projects/a09b67b513264ee9afea8afddd11a594/users/7ac0a11e964e4132b5d9c7420e661225/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:33.762170 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbc5b50-381d-4a04-b573-c62b21736286 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a09b67b513264ee9afea8afddd11a594, user_id=7ac0a11e964e4132b5d9c7420e661225, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:33.765808 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbc5b50-381d-4a04-b573-c62b21736286 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:33.779305 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1227/2477] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:30:33 2022] PUT /identity/v3/projects/a09b67b513264ee9afea8afddd11a594/users/7ac0a11e964e4132b5d9c7420e661225/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:33.790610 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd21d9d-39ef-4e9a-9892-cec0c3e4a310 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:33.790910 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd21d9d-39ef-4e9a-9892-cec0c3e4a310 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:33.791163 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd21d9d-39ef-4e9a-9892-cec0c3e4a310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:33.827010 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-7fd21d9d-39ef-4e9a-9892-cec0c3e4a310 None None] MFA Rules not processed for user `7ac0a11e964e4132b5d9c7420e661225`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:33.828700 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-7fd21d9d-39ef-4e9a-9892-cec0c3e4a310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:33.861485 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:33.929000 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1251/2478] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:33 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:33.940278 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc5b50-381d-4a04-b573-c62b21736286 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:33.942243 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-0bbc5b50-381d-4a04-b573-c62b21736286 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:34.015078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc5b50-381d-4a04-b573-c62b21736286 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:34.020447 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2376c-d381-46bf-abb1-916f44d140a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:34.021287 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2376c-d381-46bf-abb1-916f44d140a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:34.021639 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2376c-d381-46bf-abb1-916f44d140a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:34.022044 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2376c-d381-46bf-abb1-916f44d140a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:34.032842 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff2376c-d381-46bf-abb1-916f44d140a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:34.034422 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff2376c-d381-46bf-abb1-916f44d140a0 service cinder] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:34.050056 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1228/2479] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:30:33 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:30:34.159977 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd21d9d-39ef-4e9a-9892-cec0c3e4a310 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:34.170655 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd21d9d-39ef-4e9a-9892-cec0c3e4a310 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:34.174670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b4820-6f0a-4641-8ee8-88d7fa72a6cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:34.175333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b4820-6f0a-4641-8ee8-88d7fa72a6cb service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:34.175665 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b4820-6f0a-4641-8ee8-88d7fa72a6cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:34.176013 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b4820-6f0a-4641-8ee8-88d7fa72a6cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:34.283663 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7b4820-6f0a-4641-8ee8-88d7fa72a6cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:34.286640 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7b4820-6f0a-4641-8ee8-88d7fa72a6cb service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:34.299064 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1252/2480] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:30:34 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:30:44.024580 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2376c-d381-46bf-abb1-916f44d140a0 service cinder] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:44.038498 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2376c-d381-46bf-abb1-916f44d140a0 service cinder] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:44.043050 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1a587-ae7d-49ce-a92a-1e1b181a37fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:44.043696 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1a587-ae7d-49ce-a92a-1e1b181a37fa service cinder] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:44.043978 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1a587-ae7d-49ce-a92a-1e1b181a37fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:44.044270 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1a587-ae7d-49ce-a92a-1e1b181a37fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:44.046465 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-b9f1a587-ae7d-49ce-a92a-1e1b181a37fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:44.115589 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f1a587-ae7d-49ce-a92a-1e1b181a37fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:44.117078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f1a587-ae7d-49ce-a92a-1e1b181a37fa service cinder] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:44.127170 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1229/2481] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:30:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:30:44.293422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=47 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:52.101166 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f98f0-c420-4844-9bed-fe1513937aba None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:52.101873 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f98f0-c420-4844-9bed-fe1513937aba None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:52.102110 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f98f0-c420-4844-9bed-fe1513937aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:52.123565 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-c16f98f0-c420-4844-9bed-fe1513937aba None None] MFA Rules not processed for user `01be6ea7136e45aead352146193f05ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:52.124777 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-c16f98f0-c420-4844-9bed-fe1513937aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:52.132497 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1253/2482] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:52.540285 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1a587-ae7d-49ce-a92a-1e1b181a37fa service cinder] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:52.542021 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-b9f1a587-ae7d-49ce-a92a-1e1b181a37fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:52.563072 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:52.604060 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1a587-ae7d-49ce-a92a-1e1b181a37fa service cinder] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:52.607851 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eff23d6-4fd7-426e-8c42-dc1f0e44f7f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:52.608552 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff23d6-4fd7-426e-8c42-dc1f0e44f7f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:52.608829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff23d6-4fd7-426e-8c42-dc1f0e44f7f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:52.609129 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff23d6-4fd7-426e-8c42-dc1f0e44f7f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:52.611238 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-7eff23d6-4fd7-426e-8c42-dc1f0e44f7f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:52.676930 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eff23d6-4fd7-426e-8c42-dc1f0e44f7f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:52.678411 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eff23d6-4fd7-426e-8c42-dc1f0e44f7f2 service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:52.689671 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1230/2483] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:30:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:30:53.085923 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16f98f0-c420-4844-9bed-fe1513937aba None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:53.098441 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16f98f0-c420-4844-9bed-fe1513937aba None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:53.106583 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a9b70-e1cb-4a69-a5a8-1fff9e945b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:53.108102 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a9b70-e1cb-4a69-a5a8-1fff9e945b3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:53.108382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a9b70-e1cb-4a69-a5a8-1fff9e945b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:53.108652 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a9b70-e1cb-4a69-a5a8-1fff9e945b3b admin admin] PATH_INFO: `/v3/users/4a4c1aca3ff2459d8c9585c163283332` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:53.110181 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a9b70-e1cb-4a69-a5a8-1fff9e945b3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a4c1aca3ff2459d8c9585c163283332)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:53.112311 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a9b70-e1cb-4a69-a5a8-1fff9e945b3b admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:53.176098 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-625a9b70-e1cb-4a69-a5a8-1fff9e945b3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a4c1aca3ff2459d8c9585c163283332'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:53.177406 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-625a9b70-e1cb-4a69-a5a8-1fff9e945b3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a4c1aca3ff2459d8c9585c163283332'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:53.188027 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-625a9b70-e1cb-4a69-a5a8-1fff9e945b3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a4c1aca3ff2459d8c9585c163283332'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:53.189577 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-625a9b70-e1cb-4a69-a5a8-1fff9e945b3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a4c1aca3ff2459d8c9585c163283332'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:53.208226 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-625a9b70-e1cb-4a69-a5a8-1fff9e945b3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a4c1aca3ff2459d8c9585c163283332'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:53.226217 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1254/2484] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:30:53 2022] DELETE /identity/v3/users/4a4c1aca3ff2459d8c9585c163283332 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:53.231149 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a64cf0-687e-4b85-92f3-6715f7768f75 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:53.231515 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a64cf0-687e-4b85-92f3-6715f7768f75 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:53.231859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a64cf0-687e-4b85-92f3-6715f7768f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:53.250152 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-c4a64cf0-687e-4b85-92f3-6715f7768f75 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:53.251568 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-c4a64cf0-687e-4b85-92f3-6715f7768f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:53.345689 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1231/2485] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:53.355396 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a9b70-e1cb-4a69-a5a8-1fff9e945b3b admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:53.357871 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-625a9b70-e1cb-4a69-a5a8-1fff9e945b3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:53.454172 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a9b70-e1cb-4a69-a5a8-1fff9e945b3b admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:53.457708 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a8b9c-cb54-4f54-ab0e-2fb283533b7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:53.458361 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a8b9c-cb54-4f54-ab0e-2fb283533b7f service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:53.458788 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a8b9c-cb54-4f54-ab0e-2fb283533b7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:53.459063 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a8b9c-cb54-4f54-ab0e-2fb283533b7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:53.473268 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a8b9c-cb54-4f54-ab0e-2fb283533b7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:53.475209 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a8b9c-cb54-4f54-ab0e-2fb283533b7f service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:53.502896 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1255/2486] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:30:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:30:54.290074 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a64cf0-687e-4b85-92f3-6715f7768f75 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:54.292192 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-c4a64cf0-687e-4b85-92f3-6715f7768f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:54.313716 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a64cf0-687e-4b85-92f3-6715f7768f75 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:54.321199 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89799607-b569-4076-825f-011e13726225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:54.324356 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89799607-b569-4076-825f-011e13726225 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:54.324927 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89799607-b569-4076-825f-011e13726225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:54.325466 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89799607-b569-4076-825f-011e13726225 admin admin] PATH_INFO: `/v3/projects/226d9eeec7ef4e98b2b408b17deaf8af` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:54.327486 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89799607-b569-4076-825f-011e13726225 admin admin] RBAC: Authorizing `identity:delete_project(project_id=226d9eeec7ef4e98b2b408b17deaf8af)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:54.330680 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89799607-b569-4076-825f-011e13726225 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:54.386883 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-89799607-b569-4076-825f-011e13726225 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '226d9eeec7ef4e98b2b408b17deaf8af', 'request_id': 'req-89799607-b569-4076-825f-011e13726225'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:54.400308 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-89799607-b569-4076-825f-011e13726225 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '226d9eeec7ef4e98b2b408b17deaf8af', 'request_id': 'req-89799607-b569-4076-825f-011e13726225'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:54.412488 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-89799607-b569-4076-825f-011e13726225 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '226d9eeec7ef4e98b2b408b17deaf8af', 'request_id': 'req-89799607-b569-4076-825f-011e13726225'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:54.415944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:54.442453 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-89799607-b569-4076-825f-011e13726225 admin admin] The token cache is being invalidate because project 226d9eeec7ef4e98b2b408b17deaf8af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:30:54.443235 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-89799607-b569-4076-825f-011e13726225 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:54.444629 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-89799607-b569-4076-825f-011e13726225 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:54.447382 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1232/2487] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:30:54 2022] DELETE /identity/v3/projects/226d9eeec7ef4e98b2b408b17deaf8af => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:54.451218 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a8b9c-cb54-4f54-ab0e-2fb283533b7f service neutron] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:54.465571 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a8b9c-cb54-4f54-ab0e-2fb283533b7f service neutron] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:54.542770 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c08fd-8708-4b28-b79b-d4aaecb9e020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:54.543777 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c08fd-8708-4b28-b79b-d4aaecb9e020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:54.544288 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c08fd-8708-4b28-b79b-d4aaecb9e020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:54.544789 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c08fd-8708-4b28-b79b-d4aaecb9e020 admin admin] PATH_INFO: `/v3/users/7ac0a11e964e4132b5d9c7420e661225` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:54.546593 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441c08fd-8708-4b28-b79b-d4aaecb9e020 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ac0a11e964e4132b5d9c7420e661225)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:54.548854 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441c08fd-8708-4b28-b79b-d4aaecb9e020 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:54.561852 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:54.615293 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-441c08fd-8708-4b28-b79b-d4aaecb9e020 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ac0a11e964e4132b5d9c7420e661225'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:54.615975 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-441c08fd-8708-4b28-b79b-d4aaecb9e020 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ac0a11e964e4132b5d9c7420e661225'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:54.626388 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-441c08fd-8708-4b28-b79b-d4aaecb9e020 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ac0a11e964e4132b5d9c7420e661225'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:54.630374 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-441c08fd-8708-4b28-b79b-d4aaecb9e020 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ac0a11e964e4132b5d9c7420e661225'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:54.657556 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-441c08fd-8708-4b28-b79b-d4aaecb9e020 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ac0a11e964e4132b5d9c7420e661225'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:54.672381 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1256/2488] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:30:54 2022] DELETE /identity/v3/users/7ac0a11e964e4132b5d9c7420e661225 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:55.026216 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89799607-b569-4076-825f-011e13726225 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:55.028288 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-89799607-b569-4076-825f-011e13726225 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:55.050844 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89799607-b569-4076-825f-011e13726225 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:55.112526 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c44672-9eca-47b3-b57f-fdac81be2317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:55.113382 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c44672-9eca-47b3-b57f-fdac81be2317 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:55.113682 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c44672-9eca-47b3-b57f-fdac81be2317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:55.113954 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c44672-9eca-47b3-b57f-fdac81be2317 admin admin] PATH_INFO: `/v3/projects/a09b67b513264ee9afea8afddd11a594` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:55.115181 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c44672-9eca-47b3-b57f-fdac81be2317 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a09b67b513264ee9afea8afddd11a594)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:55.117138 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c44672-9eca-47b3-b57f-fdac81be2317 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:55.173233 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-96c44672-9eca-47b3-b57f-fdac81be2317 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a09b67b513264ee9afea8afddd11a594', 'request_id': 'req-96c44672-9eca-47b3-b57f-fdac81be2317'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:55.180277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-96c44672-9eca-47b3-b57f-fdac81be2317 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a09b67b513264ee9afea8afddd11a594', 'request_id': 'req-96c44672-9eca-47b3-b57f-fdac81be2317'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:55.186564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-96c44672-9eca-47b3-b57f-fdac81be2317 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a09b67b513264ee9afea8afddd11a594', 'request_id': 'req-96c44672-9eca-47b3-b57f-fdac81be2317'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:55.202882 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-96c44672-9eca-47b3-b57f-fdac81be2317 admin admin] The token cache is being invalidate because project a09b67b513264ee9afea8afddd11a594 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:30:55.203170 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-96c44672-9eca-47b3-b57f-fdac81be2317 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:55.203821 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-96c44672-9eca-47b3-b57f-fdac81be2317 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:55.205462 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1233/2489] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:30:55 2022] DELETE /identity/v3/projects/a09b67b513264ee9afea8afddd11a594 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:55.210680 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d08d5-e570-40b0-ad7b-df46fdd70b64 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:55.211043 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d08d5-e570-40b0-ad7b-df46fdd70b64 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:55.211386 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d08d5-e570-40b0-ad7b-df46fdd70b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:55.230120 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-d68d08d5-e570-40b0-ad7b-df46fdd70b64 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:55.231451 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-d68d08d5-e570-40b0-ad7b-df46fdd70b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:55.313737 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1257/2490] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:55.322872 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cafc2e4-fd88-4220-959a-937e53aa0efb None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:55.323516 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cafc2e4-fd88-4220-959a-937e53aa0efb None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:55.323516 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cafc2e4-fd88-4220-959a-937e53aa0efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:55.344073 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-1cafc2e4-fd88-4220-959a-937e53aa0efb None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:55.345355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-1cafc2e4-fd88-4220-959a-937e53aa0efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:55.356838 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1234/2491] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:55.358337 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d08d5-e570-40b0-ad7b-df46fdd70b64 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:55.369024 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68d08d5-e570-40b0-ad7b-df46fdd70b64 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:55.377186 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9679e-e8aa-4c86-9ab7-83d0f1c88911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:55.377842 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9679e-e8aa-4c86-9ab7-83d0f1c88911 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:55.378117 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9679e-e8aa-4c86-9ab7-83d0f1c88911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:55.378384 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9679e-e8aa-4c86-9ab7-83d0f1c88911 admin admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:55.379113 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d9679e-e8aa-4c86-9ab7-83d0f1c88911 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:55.379419 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d9679e-e8aa-4c86-9ab7-83d0f1c88911 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:55.381612 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d9679e-e8aa-4c86-9ab7-83d0f1c88911 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:55.398593 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1258/2492] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:30:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:55.400168 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cafc2e4-fd88-4220-959a-937e53aa0efb None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:55.411369 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cafc2e4-fd88-4220-959a-937e53aa0efb None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:55.418106 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdadd21-d364-485a-911e-079ae99fc527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:55.418864 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdadd21-d364-485a-911e-079ae99fc527 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:55.419128 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdadd21-d364-485a-911e-079ae99fc527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:55.419398 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdadd21-d364-485a-911e-079ae99fc527 admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:55.420091 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdadd21-d364-485a-911e-079ae99fc527 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:55.422051 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdadd21-d364-485a-911e-079ae99fc527 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:55.465456 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1235/2493] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:30:55 2022] POST /identity/v3/projects => generated 387 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:55.466581 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9679e-e8aa-4c86-9ab7-83d0f1c88911 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:55.487726 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9679e-e8aa-4c86-9ab7-83d0f1c88911 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:55.581187 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea3564b-ddb6-4d20-ad5b-87be347c35f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:55.582246 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3564b-ddb6-4d20-ad5b-87be347c35f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:55.582730 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3564b-ddb6-4d20-ad5b-87be347c35f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:55.583256 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea3564b-ddb6-4d20-ad5b-87be347c35f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:55.584184 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea3564b-ddb6-4d20-ad5b-87be347c35f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:55.586631 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea3564b-ddb6-4d20-ad5b-87be347c35f8 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:55.617327 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1259/2494] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:30:55 2022] POST /identity/v3/users => generated 294 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:30:55.621733 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdadd21-d364-485a-911e-079ae99fc527 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:55.635735 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdadd21-d364-485a-911e-079ae99fc527 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:55.641447 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe42cd6-d9a5-4a44-8802-b28a6fb0c983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:55.642585 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe42cd6-d9a5-4a44-8802-b28a6fb0c983 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:55.643106 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe42cd6-d9a5-4a44-8802-b28a6fb0c983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:55.643600 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe42cd6-d9a5-4a44-8802-b28a6fb0c983 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:55.644608 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe42cd6-d9a5-4a44-8802-b28a6fb0c983 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:55.646947 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe42cd6-d9a5-4a44-8802-b28a6fb0c983 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:55.668679 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1236/2495] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:55.669439 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea3564b-ddb6-4d20-ad5b-87be347c35f8 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:55.681130 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:55.690216 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea3564b-ddb6-4d20-ad5b-87be347c35f8 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:55.704108 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede0c91-1d26-4b5b-8f18-5010956fe574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:55.705130 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede0c91-1d26-4b5b-8f18-5010956fe574 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:55.705594 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede0c91-1d26-4b5b-8f18-5010956fe574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:55.706041 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede0c91-1d26-4b5b-8f18-5010956fe574 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:55.706926 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede0c91-1d26-4b5b-8f18-5010956fe574 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:55.709621 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede0c91-1d26-4b5b-8f18-5010956fe574 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:55.730754 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1260/2496] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:30:55.736631 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe42cd6-d9a5-4a44-8802-b28a6fb0c983 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:55.749310 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe42cd6-d9a5-4a44-8802-b28a6fb0c983 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:55.757682 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b8330-213d-4b2d-881d-385c75f01538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:55.759243 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b8330-213d-4b2d-881d-385c75f01538 admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:55.760003 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b8330-213d-4b2d-881d-385c75f01538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:55.760297 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3b8330-213d-4b2d-881d-385c75f01538 admin admin] PATH_INFO: `/v3/projects/cebfe8ed3e584a20b70bcaada12a2479/users/cb41863ac20146eba2ba178b6744d692/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:55.784556 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3b8330-213d-4b2d-881d-385c75f01538 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cebfe8ed3e584a20b70bcaada12a2479, user_id=cb41863ac20146eba2ba178b6744d692, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:55.788763 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3b8330-213d-4b2d-881d-385c75f01538 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:55.821378 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1237/2497] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:30:55 2022] PUT /identity/v3/projects/cebfe8ed3e584a20b70bcaada12a2479/users/cb41863ac20146eba2ba178b6744d692/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:55.824293 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099075c-abf9-4c55-9f0f-c95fbfc72ee1 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:55.824690 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099075c-abf9-4c55-9f0f-c95fbfc72ee1 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:55.825074 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8099075c-abf9-4c55-9f0f-c95fbfc72ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:55.866391 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-8099075c-abf9-4c55-9f0f-c95fbfc72ee1 None None] MFA Rules not processed for user `cb41863ac20146eba2ba178b6744d692`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:55.867833 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-8099075c-abf9-4c55-9f0f-c95fbfc72ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:55.999078 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1261/2498] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:56.000046 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b8330-213d-4b2d-881d-385c75f01538 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:56.054182 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:56.141176 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5fa73-39c6-48e5-b6bc-6260eae5b93e None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:56.141458 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5fa73-39c6-48e5-b6bc-6260eae5b93e None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:56.141916 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c5fa73-39c6-48e5-b6bc-6260eae5b93e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:56.154022 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3b8330-213d-4b2d-881d-385c75f01538 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:56.161696 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb80e83-9e8e-4b85-bdc4-e76e29bcea15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:56.162783 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb80e83-9e8e-4b85-bdc4-e76e29bcea15 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:56.163073 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb80e83-9e8e-4b85-bdc4-e76e29bcea15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:56.163351 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb80e83-9e8e-4b85-bdc4-e76e29bcea15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:56.164628 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb80e83-9e8e-4b85-bdc4-e76e29bcea15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:56.167207 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb80e83-9e8e-4b85-bdc4-e76e29bcea15 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:56.179327 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-e3c5fa73-39c6-48e5-b6bc-6260eae5b93e None None] MFA Rules not processed for user `aa92e046c3274a29a8aa8857d7570750`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:56.183848 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-e3c5fa73-39c6-48e5-b6bc-6260eae5b93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:56.204912 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1238/2499] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:30:55 2022] POST /identity/v3/projects => generated 387 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:30:56.208660 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb80e83-9e8e-4b85-bdc4-e76e29bcea15 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:56.291161 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb80e83-9e8e-4b85-bdc4-e76e29bcea15 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:56.297549 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531481dd-a582-48c2-8c6d-c573e439e53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:56.300532 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531481dd-a582-48c2-8c6d-c573e439e53e admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:56.300938 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531481dd-a582-48c2-8c6d-c573e439e53e admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:56.301463 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531481dd-a582-48c2-8c6d-c573e439e53e admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:56.302413 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531481dd-a582-48c2-8c6d-c573e439e53e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:56.304495 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531481dd-a582-48c2-8c6d-c573e439e53e admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:56.332533 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1239/2500] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:30:56 2022] POST /identity/v3/users => generated 294 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:30:56.336018 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531481dd-a582-48c2-8c6d-c573e439e53e admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:56.346919 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1262/2501] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:30:56.355137 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531481dd-a582-48c2-8c6d-c573e439e53e admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:56.361110 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84bd4f4-cb76-4b48-b2b0-b0bf6d22856f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:56.362207 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bd4f4-cb76-4b48-b2b0-b0bf6d22856f admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:56.362907 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bd4f4-cb76-4b48-b2b0-b0bf6d22856f admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:56.363399 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bd4f4-cb76-4b48-b2b0-b0bf6d22856f admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:56.365386 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84bd4f4-cb76-4b48-b2b0-b0bf6d22856f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:56.367428 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84bd4f4-cb76-4b48-b2b0-b0bf6d22856f admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:56.383261 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1240/2502] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:56.387213 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c5fa73-39c6-48e5-b6bc-6260eae5b93e None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:56.401035 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c5fa73-39c6-48e5-b6bc-6260eae5b93e None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:56.408935 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b5fa1-9f5e-4b2c-8a94-016ee871f8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:56.409755 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b5fa1-9f5e-4b2c-8a94-016ee871f8ab admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:56.410125 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b5fa1-9f5e-4b2c-8a94-016ee871f8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:56.410495 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b5fa1-9f5e-4b2c-8a94-016ee871f8ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:56.411322 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b5fa1-9f5e-4b2c-8a94-016ee871f8ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:56.413204 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b5fa1-9f5e-4b2c-8a94-016ee871f8ab admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:56.425536 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1263/2503] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:56.429848 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84bd4f4-cb76-4b48-b2b0-b0bf6d22856f admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:56.452830 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84bd4f4-cb76-4b48-b2b0-b0bf6d22856f admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:56.458166 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaaa680-ece8-404e-a894-42f70841d533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:56.459364 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaaa680-ece8-404e-a894-42f70841d533 admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:56.460046 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaaa680-ece8-404e-a894-42f70841d533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:56.460681 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaaa680-ece8-404e-a894-42f70841d533 admin admin] PATH_INFO: `/v3/projects/a0d53d48748142b0af90b27ac6e47cd6/users/9d46a2bdff074a02bfe8d489711d0040/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:56.508518 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaaa680-ece8-404e-a894-42f70841d533 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0d53d48748142b0af90b27ac6e47cd6, user_id=9d46a2bdff074a02bfe8d489711d0040, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:56.517903 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaaa680-ece8-404e-a894-42f70841d533 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:56.542706 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1241/2504] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:30:56 2022] PUT /identity/v3/projects/a0d53d48748142b0af90b27ac6e47cd6/users/9d46a2bdff074a02bfe8d489711d0040/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:30:56.555083 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfb2d8-4aeb-4ebe-8e3e-1e082796c6dd None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:56.555614 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfb2d8-4aeb-4ebe-8e3e-1e082796c6dd None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:56.555719 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfb2d8-4aeb-4ebe-8e3e-1e082796c6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:56.602834 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-cfdfb2d8-4aeb-4ebe-8e3e-1e082796c6dd None None] MFA Rules not processed for user `9d46a2bdff074a02bfe8d489711d0040`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:56.606072 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-cfdfb2d8-4aeb-4ebe-8e3e-1e082796c6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:56.704829 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1264/2505] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:56.709606 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaaa680-ece8-404e-a894-42f70841d533 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:56.777798 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:56.782392 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaaa680-ece8-404e-a894-42f70841d533 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:56.789290 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4838694f-36b9-4dd4-8e71-83599640c034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:56.789734 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4838694f-36b9-4dd4-8e71-83599640c034 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:56.790097 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4838694f-36b9-4dd4-8e71-83599640c034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:56.790621 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4838694f-36b9-4dd4-8e71-83599640c034 admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:56.791677 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4838694f-36b9-4dd4-8e71-83599640c034 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:56.794051 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4838694f-36b9-4dd4-8e71-83599640c034 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:56.835326 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1242/2506] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:30:56 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:56.836908 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdfb2d8-4aeb-4ebe-8e3e-1e082796c6dd None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:56.850333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdfb2d8-4aeb-4ebe-8e3e-1e082796c6dd None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:56.855074 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:56.953052 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab2c8b3-50f8-4fbf-bd00-c74019d0a704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:56.953908 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab2c8b3-50f8-4fbf-bd00-c74019d0a704 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:56.954169 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab2c8b3-50f8-4fbf-bd00-c74019d0a704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:56.954427 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab2c8b3-50f8-4fbf-bd00-c74019d0a704 admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:56.955183 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab2c8b3-50f8-4fbf-bd00-c74019d0a704 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:56.957262 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab2c8b3-50f8-4fbf-bd00-c74019d0a704 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:57.006580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1265/2507] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:30:56 2022] POST /identity/v3/users => generated 299 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:30:57.009205 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4838694f-36b9-4dd4-8e71-83599640c034 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:57.024753 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4838694f-36b9-4dd4-8e71-83599640c034 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:57.033362 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b74fac2-a71b-429e-ac50-53621ff83330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:57.034337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74fac2-a71b-429e-ac50-53621ff83330 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:57.035105 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74fac2-a71b-429e-ac50-53621ff83330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:57.036002 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74fac2-a71b-429e-ac50-53621ff83330 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:57.040687 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b74fac2-a71b-429e-ac50-53621ff83330 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:57.042711 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b74fac2-a71b-429e-ac50-53621ff83330 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:57.054888 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1243/2508] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:30:57 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:30:57.059708 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab2c8b3-50f8-4fbf-bd00-c74019d0a704 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:57.072714 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab2c8b3-50f8-4fbf-bd00-c74019d0a704 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:57.078880 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2413b7-862e-43a5-8851-d2caf84e5234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:57.079899 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2413b7-862e-43a5-8851-d2caf84e5234 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:57.080091 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2413b7-862e-43a5-8851-d2caf84e5234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:57.082156 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2413b7-862e-43a5-8851-d2caf84e5234 admin admin] PATH_INFO: `/v3/projects/e0efcd38aae442aaa6da959c75411d75/users/28d9fa11ccae404990539ad5185b1295/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:57.099718 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2413b7-862e-43a5-8851-d2caf84e5234 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0efcd38aae442aaa6da959c75411d75, user_id=28d9fa11ccae404990539ad5185b1295, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:57.102591 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2413b7-862e-43a5-8851-d2caf84e5234 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:57.128163 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1266/2509] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:30:57 2022] PUT /identity/v3/projects/e0efcd38aae442aaa6da959c75411d75/users/28d9fa11ccae404990539ad5185b1295/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:57.129991 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b74fac2-a71b-429e-ac50-53621ff83330 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:57.145068 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b74fac2-a71b-429e-ac50-53621ff83330 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:57.223964 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f1ea72-68a9-4fd4-ae6d-829c5221e329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:57.224777 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1ea72-68a9-4fd4-ae6d-829c5221e329 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:57.225201 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1ea72-68a9-4fd4-ae6d-829c5221e329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:57.225570 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1ea72-68a9-4fd4-ae6d-829c5221e329 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:57.226394 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f1ea72-68a9-4fd4-ae6d-829c5221e329 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:57.228499 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f1ea72-68a9-4fd4-ae6d-829c5221e329 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:57.243370 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1244/2510] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:30:57 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:57.247519 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2413b7-862e-43a5-8851-d2caf84e5234 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:57.262745 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2413b7-862e-43a5-8851-d2caf84e5234 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:57.274730 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b26bf31-b769-4558-a0fd-9fcf78500c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:57.275677 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26bf31-b769-4558-a0fd-9fcf78500c20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:57.275957 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26bf31-b769-4558-a0fd-9fcf78500c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:57.276235 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b26bf31-b769-4558-a0fd-9fcf78500c20 admin admin] PATH_INFO: `/v3/projects/e0efcd38aae442aaa6da959c75411d75/users/28d9fa11ccae404990539ad5185b1295/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:57.279624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b26bf31-b769-4558-a0fd-9fcf78500c20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0efcd38aae442aaa6da959c75411d75, user_id=28d9fa11ccae404990539ad5185b1295, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:57.285741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b26bf31-b769-4558-a0fd-9fcf78500c20 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:57.304389 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1267/2511] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:30:57 2022] PUT /identity/v3/projects/e0efcd38aae442aaa6da959c75411d75/users/28d9fa11ccae404990539ad5185b1295/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:57.316481 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15779962-1c7b-476f-be4a-ab897faea53a None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:57.316806 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15779962-1c7b-476f-be4a-ab897faea53a None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:57.317079 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15779962-1c7b-476f-be4a-ab897faea53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:57.385654 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-15779962-1c7b-476f-be4a-ab897faea53a None None] MFA Rules not processed for user `28d9fa11ccae404990539ad5185b1295`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:57.389748 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-15779962-1c7b-476f-be4a-ab897faea53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:57.579217 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1245/2512] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:57 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:57.580519 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b26bf31-b769-4558-a0fd-9fcf78500c20 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:57.587363 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-7b26bf31-b769-4558-a0fd-9fcf78500c20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:57.780872 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b26bf31-b769-4558-a0fd-9fcf78500c20 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:57.793542 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384b3b77-307a-40db-beac-d6305571c781 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:57.794230 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b3b77-307a-40db-beac-d6305571c781 service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:57.794512 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b3b77-307a-40db-beac-d6305571c781 service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:57.794787 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b3b77-307a-40db-beac-d6305571c781 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:57.815121 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384b3b77-307a-40db-beac-d6305571c781 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:57.816598 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384b3b77-307a-40db-beac-d6305571c781 service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:57.830061 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1268/2513] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:30:57 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:30:58.254639 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15779962-1c7b-476f-be4a-ab897faea53a None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:58.256832 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-15779962-1c7b-476f-be4a-ab897faea53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:58.316255 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:58.331665 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15779962-1c7b-476f-be4a-ab897faea53a None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:58.337617 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d836cd8-b7c4-43bc-a861-2d9593bd26b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:58.338654 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d836cd8-b7c4-43bc-a861-2d9593bd26b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:58.338940 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d836cd8-b7c4-43bc-a861-2d9593bd26b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:58.339219 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d836cd8-b7c4-43bc-a861-2d9593bd26b0 admin admin] PATH_INFO: `/v3/users/a094273e734342eca2bdf159fab8a8d2` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:58.340538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d836cd8-b7c4-43bc-a861-2d9593bd26b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a094273e734342eca2bdf159fab8a8d2)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:58.342552 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d836cd8-b7c4-43bc-a861-2d9593bd26b0 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:58.391627 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-3d836cd8-b7c4-43bc-a861-2d9593bd26b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a094273e734342eca2bdf159fab8a8d2'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:58.407471 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-3d836cd8-b7c4-43bc-a861-2d9593bd26b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a094273e734342eca2bdf159fab8a8d2'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:58.412714 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-3d836cd8-b7c4-43bc-a861-2d9593bd26b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a094273e734342eca2bdf159fab8a8d2'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:58.420156 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-3d836cd8-b7c4-43bc-a861-2d9593bd26b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a094273e734342eca2bdf159fab8a8d2'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:58.420841 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-3d836cd8-b7c4-43bc-a861-2d9593bd26b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a094273e734342eca2bdf159fab8a8d2'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:58.426738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1246/2514] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:30:58 2022] DELETE /identity/v3/users/a094273e734342eca2bdf159fab8a8d2 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:30:58.531758 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384b3b77-307a-40db-beac-d6305571c781 service cinder] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:58.533189 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-384b3b77-307a-40db-beac-d6305571c781 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:58.546244 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384b3b77-307a-40db-beac-d6305571c781 service cinder] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:58.625649 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747f50fc-8c44-4eea-bbaa-6790aa3e4bea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:58.631465 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f50fc-8c44-4eea-bbaa-6790aa3e4bea service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:58.631755 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f50fc-8c44-4eea-bbaa-6790aa3e4bea service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:58.632035 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747f50fc-8c44-4eea-bbaa-6790aa3e4bea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:58.634637 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-747f50fc-8c44-4eea-bbaa-6790aa3e4bea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:58.656625 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d836cd8-b7c4-43bc-a861-2d9593bd26b0 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:58.658927 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-3d836cd8-b7c4-43bc-a861-2d9593bd26b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:58.686407 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:58.786278 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747f50fc-8c44-4eea-bbaa-6790aa3e4bea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:58.789526 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747f50fc-8c44-4eea-bbaa-6790aa3e4bea service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:58.796332 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d836cd8-b7c4-43bc-a861-2d9593bd26b0 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:58.804525 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1269/2515] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:30:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:30:58.809856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aabb85-7736-42bd-9dad-52ae1308b731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:58.810746 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aabb85-7736-42bd-9dad-52ae1308b731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:58.811037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aabb85-7736-42bd-9dad-52ae1308b731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:58.811276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aabb85-7736-42bd-9dad-52ae1308b731 admin admin] PATH_INFO: `/v3/projects/082b5523ee504be09f6fcaf833207a4f` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:58.812609 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aabb85-7736-42bd-9dad-52ae1308b731 admin admin] RBAC: Authorizing `identity:delete_project(project_id=082b5523ee504be09f6fcaf833207a4f)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:58.814602 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aabb85-7736-42bd-9dad-52ae1308b731 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:58.884566 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-70aabb85-7736-42bd-9dad-52ae1308b731 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '082b5523ee504be09f6fcaf833207a4f', 'request_id': 'req-70aabb85-7736-42bd-9dad-52ae1308b731'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:58.891062 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-70aabb85-7736-42bd-9dad-52ae1308b731 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '082b5523ee504be09f6fcaf833207a4f', 'request_id': 'req-70aabb85-7736-42bd-9dad-52ae1308b731'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:58.896733 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-70aabb85-7736-42bd-9dad-52ae1308b731 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '082b5523ee504be09f6fcaf833207a4f', 'request_id': 'req-70aabb85-7736-42bd-9dad-52ae1308b731'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:58.909970 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-70aabb85-7736-42bd-9dad-52ae1308b731 admin admin] The token cache is being invalidate because project 082b5523ee504be09f6fcaf833207a4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:30:58.910254 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-70aabb85-7736-42bd-9dad-52ae1308b731 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:58.910872 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-70aabb85-7736-42bd-9dad-52ae1308b731 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:30:58.912478 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1247/2516] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:30:58 2022] DELETE /identity/v3/projects/082b5523ee504be09f6fcaf833207a4f => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:58.917824 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3252d-4a4a-4ee8-8f3f-66fcbbd392a3 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:58.918301 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3252d-4a4a-4ee8-8f3f-66fcbbd392a3 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:58.918645 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3252d-4a4a-4ee8-8f3f-66fcbbd392a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:58.937616 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-11a3252d-4a4a-4ee8-8f3f-66fcbbd392a3 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:58.938834 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-11a3252d-4a4a-4ee8-8f3f-66fcbbd392a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:59.006148 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1270/2517] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:59.015101 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f9c59-bce6-4350-a368-6594a3a41074 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:59.015456 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f9c59-bce6-4350-a368-6594a3a41074 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:59.015708 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f9c59-bce6-4350-a368-6594a3a41074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:59.022712 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:59.040676 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-7c8f9c59-bce6-4350-a368-6594a3a41074 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:59.041985 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-7c8f9c59-bce6-4350-a368-6594a3a41074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:59.062556 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1248/2518] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:59.062695 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a3252d-4a4a-4ee8-8f3f-66fcbbd392a3 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:59.071930 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a3252d-4a4a-4ee8-8f3f-66fcbbd392a3 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:59.077127 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd4502-3f72-4625-b0fc-af31a956ef73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:59.077898 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd4502-3f72-4625-b0fc-af31a956ef73 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:59.078243 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd4502-3f72-4625-b0fc-af31a956ef73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:59.078583 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd4502-3f72-4625-b0fc-af31a956ef73 admin admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:59.079386 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd4502-3f72-4625-b0fc-af31a956ef73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:30:59.079760 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd4502-3f72-4625-b0fc-af31a956ef73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:59.081496 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd4502-3f72-4625-b0fc-af31a956ef73 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:59.101032 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1271/2519] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:30:59.103402 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8f9c59-bce6-4350-a368-6594a3a41074 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:59.117043 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8f9c59-bce6-4350-a368-6594a3a41074 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:59.128547 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11b298-c99d-44f9-8dbc-cf04aa49b840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:59.128547 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11b298-c99d-44f9-8dbc-cf04aa49b840 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:59.128854 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11b298-c99d-44f9-8dbc-cf04aa49b840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:59.129090 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11b298-c99d-44f9-8dbc-cf04aa49b840 admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:59.130385 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a11b298-c99d-44f9-8dbc-cf04aa49b840 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:59.132676 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a11b298-c99d-44f9-8dbc-cf04aa49b840 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:59.170599 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1249/2520] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:30:59 2022] POST /identity/v3/projects => generated 367 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:59.176648 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd4502-3f72-4625-b0fc-af31a956ef73 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:59.189787 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd4502-3f72-4625-b0fc-af31a956ef73 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:59.280983 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a40541-6c9d-45bf-9360-eff3081a5c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:59.281679 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a40541-6c9d-45bf-9360-eff3081a5c4c admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:59.281955 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a40541-6c9d-45bf-9360-eff3081a5c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:59.282267 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a40541-6c9d-45bf-9360-eff3081a5c4c admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:59.283366 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a40541-6c9d-45bf-9360-eff3081a5c4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:59.285332 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a40541-6c9d-45bf-9360-eff3081a5c4c admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:59.320783 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1272/2521] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:30:59 2022] POST /identity/v3/users => generated 284 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:59.322111 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11b298-c99d-44f9-8dbc-cf04aa49b840 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:59.334209 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11b298-c99d-44f9-8dbc-cf04aa49b840 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:59.341510 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9149f8e8-b162-4f83-912b-12f1bcdff7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:59.342250 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9149f8e8-b162-4f83-912b-12f1bcdff7b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:59.342534 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9149f8e8-b162-4f83-912b-12f1bcdff7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:59.342837 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9149f8e8-b162-4f83-912b-12f1bcdff7b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:59.343748 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9149f8e8-b162-4f83-912b-12f1bcdff7b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:59.345597 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9149f8e8-b162-4f83-912b-12f1bcdff7b5 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:59.360205 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1250/2522] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:59.362801 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a40541-6c9d-45bf-9360-eff3081a5c4c admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:59.375172 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a40541-6c9d-45bf-9360-eff3081a5c4c admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:59.381414 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70083ceb-f313-4504-968e-58b88acd55a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:59.383372 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70083ceb-f313-4504-968e-58b88acd55a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:59.383666 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70083ceb-f313-4504-968e-58b88acd55a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:59.383960 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70083ceb-f313-4504-968e-58b88acd55a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:59.385366 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70083ceb-f313-4504-968e-58b88acd55a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:59.387117 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70083ceb-f313-4504-968e-58b88acd55a7 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:59.398481 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1273/2523] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:59.402718 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9149f8e8-b162-4f83-912b-12f1bcdff7b5 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:59.413981 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9149f8e8-b162-4f83-912b-12f1bcdff7b5 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:59.419401 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f29f8-1842-4c9c-99d7-8db2d56a819a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:59.421677 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f29f8-1842-4c9c-99d7-8db2d56a819a admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:59.422143 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f29f8-1842-4c9c-99d7-8db2d56a819a admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:59.422546 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f29f8-1842-4c9c-99d7-8db2d56a819a admin admin] PATH_INFO: `/v3/projects/958a87792f0b44e0b3973db8b6894476/users/96a4d4e39c7d44f9bc39c1061a09a55b/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:59.438239 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f29f8-1842-4c9c-99d7-8db2d56a819a admin admin] RBAC: Authorizing `identity:create_grant(project_id=958a87792f0b44e0b3973db8b6894476, user_id=96a4d4e39c7d44f9bc39c1061a09a55b, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:59.440917 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f29f8-1842-4c9c-99d7-8db2d56a819a admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:59.459091 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1251/2524] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:30:59 2022] PUT /identity/v3/projects/958a87792f0b44e0b3973db8b6894476/users/96a4d4e39c7d44f9bc39c1061a09a55b/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:30:59.469598 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb015279-32cc-4f8f-8ba1-57d94d862b03 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:59.469884 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb015279-32cc-4f8f-8ba1-57d94d862b03 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:59.470155 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb015279-32cc-4f8f-8ba1-57d94d862b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:59.498127 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-eb015279-32cc-4f8f-8ba1-57d94d862b03 None None] MFA Rules not processed for user `96a4d4e39c7d44f9bc39c1061a09a55b`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:30:59.499339 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-eb015279-32cc-4f8f-8ba1-57d94d862b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:30:59.509459 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:30:59.569715 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1274/2525] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:30:59 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:30:59.574711 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f29f8-1842-4c9c-99d7-8db2d56a819a admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:59.639946 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f29f8-1842-4c9c-99d7-8db2d56a819a admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:59.644740 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8326b4f1-b218-4521-9b38-c21f4eeff99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:59.645433 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8326b4f1-b218-4521-9b38-c21f4eeff99b admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:59.645714 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8326b4f1-b218-4521-9b38-c21f4eeff99b admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:59.645987 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8326b4f1-b218-4521-9b38-c21f4eeff99b admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:59.646679 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8326b4f1-b218-4521-9b38-c21f4eeff99b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:59.648585 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8326b4f1-b218-4521-9b38-c21f4eeff99b admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:59.677388 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1252/2526] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:30:59 2022] POST /identity/v3/projects => generated 367 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:59.681754 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb015279-32cc-4f8f-8ba1-57d94d862b03 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:59.693053 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb015279-32cc-4f8f-8ba1-57d94d862b03 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:59.755071 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0039c27-344e-4946-a583-2f15cdb08c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:59.756099 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0039c27-344e-4946-a583-2f15cdb08c5d admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:59.756377 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0039c27-344e-4946-a583-2f15cdb08c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:59.756646 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0039c27-344e-4946-a583-2f15cdb08c5d admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:59.757376 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0039c27-344e-4946-a583-2f15cdb08c5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:59.759460 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0039c27-344e-4946-a583-2f15cdb08c5d admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:59.780097 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1275/2527] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:30:59 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:30:59.784186 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8326b4f1-b218-4521-9b38-c21f4eeff99b admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:59.795646 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8326b4f1-b218-4521-9b38-c21f4eeff99b admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:59.800687 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7e97a-694a-4801-beb8-be553daf34b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:59.801408 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7e97a-694a-4801-beb8-be553daf34b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:59.801686 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7e97a-694a-4801-beb8-be553daf34b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:59.801960 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7e97a-694a-4801-beb8-be553daf34b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:59.802687 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b7e97a-694a-4801-beb8-be553daf34b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:59.804296 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b7e97a-694a-4801-beb8-be553daf34b5 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:59.815763 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1253/2528] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:59.820143 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0039c27-344e-4946-a583-2f15cdb08c5d admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:59.830914 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0039c27-344e-4946-a583-2f15cdb08c5d admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:59.835765 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f99edd5-4a0b-44fc-aac4-0b20512c0e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:59.836691 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f99edd5-4a0b-44fc-aac4-0b20512c0e63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:59.837112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f99edd5-4a0b-44fc-aac4-0b20512c0e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:59.837386 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f99edd5-4a0b-44fc-aac4-0b20512c0e63 admin admin] PATH_INFO: `/v3/projects/f097eb6f35a643f18da53d7e3caea123/users/5acad59d36b24cbaba4fe002b9758a7c/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:59.853219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f99edd5-4a0b-44fc-aac4-0b20512c0e63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f097eb6f35a643f18da53d7e3caea123, user_id=5acad59d36b24cbaba4fe002b9758a7c, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:59.856221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f99edd5-4a0b-44fc-aac4-0b20512c0e63 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:59.872233 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1276/2529] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:30:59 2022] PUT /identity/v3/projects/f097eb6f35a643f18da53d7e3caea123/users/5acad59d36b24cbaba4fe002b9758a7c/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:30:59.876969 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7e97a-694a-4801-beb8-be553daf34b5 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:59.888647 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7e97a-694a-4801-beb8-be553daf34b5 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:59.957134 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dfa3e0-3bab-4a3b-88f1-2403b1ebfb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:59.957819 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfa3e0-3bab-4a3b-88f1-2403b1ebfb43 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:30:59.958100 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfa3e0-3bab-4a3b-88f1-2403b1ebfb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:30:59.958377 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dfa3e0-3bab-4a3b-88f1-2403b1ebfb43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:30:59.959116 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dfa3e0-3bab-4a3b-88f1-2403b1ebfb43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:30:59.960746 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dfa3e0-3bab-4a3b-88f1-2403b1ebfb43 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:30:59.974741 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1254/2530] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:30:59.983633 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f99edd5-4a0b-44fc-aac4-0b20512c0e63 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:30:59.994435 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f99edd5-4a0b-44fc-aac4-0b20512c0e63 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:30:59.999003 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d97a2-971c-441c-9da6-bb09401bd3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:30:59.999890 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d97a2-971c-441c-9da6-bb09401bd3bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:00.000166 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d97a2-971c-441c-9da6-bb09401bd3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:00.000435 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7d97a2-971c-441c-9da6-bb09401bd3bb admin admin] PATH_INFO: `/v3/projects/f097eb6f35a643f18da53d7e3caea123/users/5acad59d36b24cbaba4fe002b9758a7c/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:00.002375 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7d97a2-971c-441c-9da6-bb09401bd3bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f097eb6f35a643f18da53d7e3caea123, user_id=5acad59d36b24cbaba4fe002b9758a7c, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:00.005047 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7d97a2-971c-441c-9da6-bb09401bd3bb admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:00.018288 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1277/2531] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:30:59 2022] PUT /identity/v3/projects/f097eb6f35a643f18da53d7e3caea123/users/5acad59d36b24cbaba4fe002b9758a7c/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:00.029981 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af2d73-6446-4984-8903-1aae7aa6e104 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:00.030308 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af2d73-6446-4984-8903-1aae7aa6e104 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:00.030571 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af2d73-6446-4984-8903-1aae7aa6e104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:00.049149 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:00.061615 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-b2af2d73-6446-4984-8903-1aae7aa6e104 None None] MFA Rules not processed for user `5acad59d36b24cbaba4fe002b9758a7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:00.063557 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-b2af2d73-6446-4984-8903-1aae7aa6e104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:00.149130 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1255/2532] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:00 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:00.162409 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d97a2-971c-441c-9da6-bb09401bd3bb admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:00.164158 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-6b7d97a2-971c-441c-9da6-bb09401bd3bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:00.235844 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7d97a2-971c-441c-9da6-bb09401bd3bb admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:00.245334 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7795d6-9c47-4a77-82dc-3c6f193d36f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:00.246310 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7795d6-9c47-4a77-82dc-3c6f193d36f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:00.246600 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7795d6-9c47-4a77-82dc-3c6f193d36f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:00.246881 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7795d6-9c47-4a77-82dc-3c6f193d36f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:00.259064 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7795d6-9c47-4a77-82dc-3c6f193d36f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:00.262165 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7795d6-9c47-4a77-82dc-3c6f193d36f7 service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:00.275508 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1278/2533] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:00 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:01.767430 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af2d73-6446-4984-8903-1aae7aa6e104 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:01.832789 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af2d73-6446-4984-8903-1aae7aa6e104 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:01.838095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2065a9a2-d859-4700-8b51-6cf17059ae20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:01.838988 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065a9a2-d859-4700-8b51-6cf17059ae20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:01.839266 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065a9a2-d859-4700-8b51-6cf17059ae20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:01.839538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065a9a2-d859-4700-8b51-6cf17059ae20 admin admin] PATH_INFO: `/v3/users/96a4d4e39c7d44f9bc39c1061a09a55b` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:01.840789 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2065a9a2-d859-4700-8b51-6cf17059ae20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96a4d4e39c7d44f9bc39c1061a09a55b)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:01.842763 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2065a9a2-d859-4700-8b51-6cf17059ae20 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:01.902490 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-2065a9a2-d859-4700-8b51-6cf17059ae20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96a4d4e39c7d44f9bc39c1061a09a55b'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:01.914785 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-2065a9a2-d859-4700-8b51-6cf17059ae20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96a4d4e39c7d44f9bc39c1061a09a55b'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:01.925769 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-2065a9a2-d859-4700-8b51-6cf17059ae20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96a4d4e39c7d44f9bc39c1061a09a55b'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:01.933139 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-2065a9a2-d859-4700-8b51-6cf17059ae20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96a4d4e39c7d44f9bc39c1061a09a55b'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:01.933804 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-2065a9a2-d859-4700-8b51-6cf17059ae20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96a4d4e39c7d44f9bc39c1061a09a55b'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:01.940616 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1256/2534] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:31:01 2022] DELETE /identity/v3/users/96a4d4e39c7d44f9bc39c1061a09a55b => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:31:01.945152 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191bc9d5-1e6d-41d6-bc4a-f751da5067a5 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:01.945621 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191bc9d5-1e6d-41d6-bc4a-f751da5067a5 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:01.945738 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191bc9d5-1e6d-41d6-bc4a-f751da5067a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:01.959130 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:01.966078 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-191bc9d5-1e6d-41d6-bc4a-f751da5067a5 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:01.968274 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-191bc9d5-1e6d-41d6-bc4a-f751da5067a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:02.029274 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1279/2535] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:02.039693 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2065a9a2-d859-4700-8b51-6cf17059ae20 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:02.041932 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-2065a9a2-d859-4700-8b51-6cf17059ae20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:02.059752 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:02.107919 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2065a9a2-d859-4700-8b51-6cf17059ae20 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:02.112176 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d9651-1019-4197-b937-e8239b1fa4c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:02.112837 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d9651-1019-4197-b937-e8239b1fa4c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:02.113150 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d9651-1019-4197-b937-e8239b1fa4c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:02.113428 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d9651-1019-4197-b937-e8239b1fa4c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:02.125733 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d9651-1019-4197-b937-e8239b1fa4c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:02.127512 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d9651-1019-4197-b937-e8239b1fa4c5 service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:02.139433 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1257/2536] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:02.434635 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191bc9d5-1e6d-41d6-bc4a-f751da5067a5 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:02.436224 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-191bc9d5-1e6d-41d6-bc4a-f751da5067a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:02.449653 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191bc9d5-1e6d-41d6-bc4a-f751da5067a5 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:02.455207 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb74a502-2776-4f0b-bf4f-392e50298ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:02.456207 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb74a502-2776-4f0b-bf4f-392e50298ebb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:02.456558 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb74a502-2776-4f0b-bf4f-392e50298ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:02.456834 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb74a502-2776-4f0b-bf4f-392e50298ebb admin admin] PATH_INFO: `/v3/projects/958a87792f0b44e0b3973db8b6894476` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:02.458193 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb74a502-2776-4f0b-bf4f-392e50298ebb admin admin] RBAC: Authorizing `identity:delete_project(project_id=958a87792f0b44e0b3973db8b6894476)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:02.460299 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb74a502-2776-4f0b-bf4f-392e50298ebb admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:02.511832 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-eb74a502-2776-4f0b-bf4f-392e50298ebb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '958a87792f0b44e0b3973db8b6894476', 'request_id': 'req-eb74a502-2776-4f0b-bf4f-392e50298ebb'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:02.518067 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-eb74a502-2776-4f0b-bf4f-392e50298ebb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '958a87792f0b44e0b3973db8b6894476', 'request_id': 'req-eb74a502-2776-4f0b-bf4f-392e50298ebb'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:02.527775 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-eb74a502-2776-4f0b-bf4f-392e50298ebb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '958a87792f0b44e0b3973db8b6894476', 'request_id': 'req-eb74a502-2776-4f0b-bf4f-392e50298ebb'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:02.539259 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-eb74a502-2776-4f0b-bf4f-392e50298ebb admin admin] The token cache is being invalidate because project 958a87792f0b44e0b3973db8b6894476 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:31:02.539547 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-eb74a502-2776-4f0b-bf4f-392e50298ebb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:02.540272 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-eb74a502-2776-4f0b-bf4f-392e50298ebb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:02.541879 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1280/2537] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:31:02 2022] DELETE /identity/v3/projects/958a87792f0b44e0b3973db8b6894476 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:02.546178 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d9651-1019-4197-b937-e8239b1fa4c5 service neutron] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:02.556041 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d9651-1019-4197-b937-e8239b1fa4c5 service neutron] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:02.619895 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1a731-1e23-4d61-9205-b0f2dafb6b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:02.620829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1a731-1e23-4d61-9205-b0f2dafb6b38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:02.621152 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1a731-1e23-4d61-9205-b0f2dafb6b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:02.621385 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1a731-1e23-4d61-9205-b0f2dafb6b38 admin admin] PATH_INFO: `/v3/users/5acad59d36b24cbaba4fe002b9758a7c` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:02.622652 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e1a731-1e23-4d61-9205-b0f2dafb6b38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5acad59d36b24cbaba4fe002b9758a7c)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:02.624641 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e1a731-1e23-4d61-9205-b0f2dafb6b38 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:02.676391 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-76e1a731-1e23-4d61-9205-b0f2dafb6b38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5acad59d36b24cbaba4fe002b9758a7c'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:02.689321 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-76e1a731-1e23-4d61-9205-b0f2dafb6b38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5acad59d36b24cbaba4fe002b9758a7c'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:02.698042 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-76e1a731-1e23-4d61-9205-b0f2dafb6b38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5acad59d36b24cbaba4fe002b9758a7c'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:02.707078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-76e1a731-1e23-4d61-9205-b0f2dafb6b38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5acad59d36b24cbaba4fe002b9758a7c'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:02.708147 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-76e1a731-1e23-4d61-9205-b0f2dafb6b38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5acad59d36b24cbaba4fe002b9758a7c'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:02.716058 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1258/2538] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:31:02 2022] DELETE /identity/v3/users/5acad59d36b24cbaba4fe002b9758a7c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:02.949624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb74a502-2776-4f0b-bf4f-392e50298ebb admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:02.951248 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-eb74a502-2776-4f0b-bf4f-392e50298ebb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:03.026555 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb74a502-2776-4f0b-bf4f-392e50298ebb admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:03.082129 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:03.090569 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4bca3c-9d1e-46a0-b087-dfc33a688276 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:03.092092 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4bca3c-9d1e-46a0-b087-dfc33a688276 service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:03.092846 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4bca3c-9d1e-46a0-b087-dfc33a688276 service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:03.093770 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4bca3c-9d1e-46a0-b087-dfc33a688276 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:03.095816 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-4b4bca3c-9d1e-46a0-b087-dfc33a688276 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:03.171437 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4bca3c-9d1e-46a0-b087-dfc33a688276 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:03.172942 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4bca3c-9d1e-46a0-b087-dfc33a688276 service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:03.184302 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1281/2539] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:03.557738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1a731-1e23-4d61-9205-b0f2dafb6b38 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:03.559401 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-76e1a731-1e23-4d61-9205-b0f2dafb6b38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:03.676672 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:03.719151 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1a731-1e23-4d61-9205-b0f2dafb6b38 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:03.725560 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd17b15-5329-4548-ba91-59aae900c8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:03.726598 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd17b15-5329-4548-ba91-59aae900c8f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:03.727041 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd17b15-5329-4548-ba91-59aae900c8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:03.727328 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd17b15-5329-4548-ba91-59aae900c8f3 admin admin] PATH_INFO: `/v3/projects/f097eb6f35a643f18da53d7e3caea123` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:03.728911 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd17b15-5329-4548-ba91-59aae900c8f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f097eb6f35a643f18da53d7e3caea123)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:03.731046 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd17b15-5329-4548-ba91-59aae900c8f3 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:03.807916 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-bcd17b15-5329-4548-ba91-59aae900c8f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f097eb6f35a643f18da53d7e3caea123', 'request_id': 'req-bcd17b15-5329-4548-ba91-59aae900c8f3'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:03.815706 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-bcd17b15-5329-4548-ba91-59aae900c8f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f097eb6f35a643f18da53d7e3caea123', 'request_id': 'req-bcd17b15-5329-4548-ba91-59aae900c8f3'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:03.822708 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-bcd17b15-5329-4548-ba91-59aae900c8f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f097eb6f35a643f18da53d7e3caea123', 'request_id': 'req-bcd17b15-5329-4548-ba91-59aae900c8f3'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:03.839440 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-bcd17b15-5329-4548-ba91-59aae900c8f3 admin admin] The token cache is being invalidate because project f097eb6f35a643f18da53d7e3caea123 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:31:03.839786 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-bcd17b15-5329-4548-ba91-59aae900c8f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:03.841084 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-bcd17b15-5329-4548-ba91-59aae900c8f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:03.843335 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1259/2540] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:31:03 2022] DELETE /identity/v3/projects/f097eb6f35a643f18da53d7e3caea123 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:03.848218 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46ebda-9786-42ed-931c-a9e26d18b581 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:03.848544 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46ebda-9786-42ed-931c-a9e26d18b581 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:03.848792 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46ebda-9786-42ed-931c-a9e26d18b581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:03.867934 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-2c46ebda-9786-42ed-931c-a9e26d18b581 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:03.869586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-2c46ebda-9786-42ed-931c-a9e26d18b581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:03.945566 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1282/2541] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:31:03.953365 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ef85d-8d13-42d2-b6a5-5999e5e03c46 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:03.953658 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ef85d-8d13-42d2-b6a5-5999e5e03c46 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:03.953955 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ef85d-8d13-42d2-b6a5-5999e5e03c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:03.973118 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-313ef85d-8d13-42d2-b6a5-5999e5e03c46 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:03.974314 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-313ef85d-8d13-42d2-b6a5-5999e5e03c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:03.982562 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1260/2542] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:03.987170 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c46ebda-9786-42ed-931c-a9e26d18b581 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:04.000452 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c46ebda-9786-42ed-931c-a9e26d18b581 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:04.005268 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e4ee5b-6a86-4f65-940e-3f356d93c75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:04.006290 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4ee5b-6a86-4f65-940e-3f356d93c75c admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:04.006569 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4ee5b-6a86-4f65-940e-3f356d93c75c admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:04.006844 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e4ee5b-6a86-4f65-940e-3f356d93c75c admin admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:04.007571 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e4ee5b-6a86-4f65-940e-3f356d93c75c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:31:04.007868 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e4ee5b-6a86-4f65-940e-3f356d93c75c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:04.010023 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e4ee5b-6a86-4f65-940e-3f356d93c75c admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:04.025867 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1283/2543] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:31:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:31:04.027046 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ef85d-8d13-42d2-b6a5-5999e5e03c46 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:04.036598 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ef85d-8d13-42d2-b6a5-5999e5e03c46 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:04.042230 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd34f9-a2fa-48c1-86ec-9a31749f646c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:04.042883 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd34f9-a2fa-48c1-86ec-9a31749f646c admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:04.043158 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd34f9-a2fa-48c1-86ec-9a31749f646c admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:04.043429 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffd34f9-a2fa-48c1-86ec-9a31749f646c admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:04.044111 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffd34f9-a2fa-48c1-86ec-9a31749f646c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:04.046098 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffd34f9-a2fa-48c1-86ec-9a31749f646c admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:04.073963 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1261/2544] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:31:04 2022] POST /identity/v3/projects => generated 399 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:31:04.078224 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e4ee5b-6a86-4f65-940e-3f356d93c75c admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:04.090016 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e4ee5b-6a86-4f65-940e-3f356d93c75c admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:04.166421 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71953c33-7c28-4869-a2b2-8ed1538557e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:04.167111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71953c33-7c28-4869-a2b2-8ed1538557e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:04.167902 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71953c33-7c28-4869-a2b2-8ed1538557e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:04.167902 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71953c33-7c28-4869-a2b2-8ed1538557e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:04.168483 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71953c33-7c28-4869-a2b2-8ed1538557e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:04.170475 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71953c33-7c28-4869-a2b2-8ed1538557e7 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:04.197586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1284/2545] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:31:04 2022] POST /identity/v3/users => generated 300 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:04.199241 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd34f9-a2fa-48c1-86ec-9a31749f646c admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:04.210312 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffd34f9-a2fa-48c1-86ec-9a31749f646c admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:04.216760 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f18aa-3fc2-41c4-baa2-de3a10f6a6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:04.217550 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f18aa-3fc2-41c4-baa2-de3a10f6a6c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:04.217900 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f18aa-3fc2-41c4-baa2-de3a10f6a6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:04.218308 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f18aa-3fc2-41c4-baa2-de3a10f6a6c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:04.219200 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f18aa-3fc2-41c4-baa2-de3a10f6a6c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:04.221217 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f18aa-3fc2-41c4-baa2-de3a10f6a6c1 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:04.232766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1262/2546] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:04.238821 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71953c33-7c28-4869-a2b2-8ed1538557e7 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:04.252061 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71953c33-7c28-4869-a2b2-8ed1538557e7 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:04.259862 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf74b2-25df-4889-9db9-765c657fa711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:04.260696 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf74b2-25df-4889-9db9-765c657fa711 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:04.261011 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf74b2-25df-4889-9db9-765c657fa711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:04.261292 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf74b2-25df-4889-9db9-765c657fa711 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:04.262057 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdf74b2-25df-4889-9db9-765c657fa711 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:04.263680 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdf74b2-25df-4889-9db9-765c657fa711 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:04.276276 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1285/2547] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:04.280641 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f18aa-3fc2-41c4-baa2-de3a10f6a6c1 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:04.291479 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f18aa-3fc2-41c4-baa2-de3a10f6a6c1 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:04.296673 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b2944-bb90-4eb4-8f22-dece21144ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:04.298482 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b2944-bb90-4eb4-8f22-dece21144ffb admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:04.298773 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b2944-bb90-4eb4-8f22-dece21144ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:04.299050 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b2944-bb90-4eb4-8f22-dece21144ffb admin admin] PATH_INFO: `/v3/projects/b3bc8b78ca6848c6a1bbd707ecb32e74/users/f2e4508850bd40deb267494189bb63dc/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:04.314387 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57b2944-bb90-4eb4-8f22-dece21144ffb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3bc8b78ca6848c6a1bbd707ecb32e74, user_id=f2e4508850bd40deb267494189bb63dc, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:04.317253 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57b2944-bb90-4eb4-8f22-dece21144ffb admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:04.333516 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1263/2548] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:31:04 2022] PUT /identity/v3/projects/b3bc8b78ca6848c6a1bbd707ecb32e74/users/f2e4508850bd40deb267494189bb63dc/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:04.344091 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0cdf9-9d70-4e87-af8e-82cd10a1c3b7 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:04.344447 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0cdf9-9d70-4e87-af8e-82cd10a1c3b7 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:04.344797 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0cdf9-9d70-4e87-af8e-82cd10a1c3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:04.358510 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:04.376898 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-4da0cdf9-9d70-4e87-af8e-82cd10a1c3b7 None None] MFA Rules not processed for user `f2e4508850bd40deb267494189bb63dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:04.378114 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-4da0cdf9-9d70-4e87-af8e-82cd10a1c3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:04.450682 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1286/2549] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:04 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:04.457577 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b2944-bb90-4eb4-8f22-dece21144ffb admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:04.527345 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b2944-bb90-4eb4-8f22-dece21144ffb admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:04.532206 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d7504-300a-4c1d-b992-119c8a153800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:04.532866 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d7504-300a-4c1d-b992-119c8a153800 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:04.533169 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d7504-300a-4c1d-b992-119c8a153800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:04.533439 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d7504-300a-4c1d-b992-119c8a153800 admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:04.534129 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d7504-300a-4c1d-b992-119c8a153800 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:04.536051 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d7504-300a-4c1d-b992-119c8a153800 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:04.540037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:04.564126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1264/2550] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:31:04 2022] POST /identity/v3/projects => generated 401 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:04.568209 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da0cdf9-9d70-4e87-af8e-82cd10a1c3b7 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:04.579272 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da0cdf9-9d70-4e87-af8e-82cd10a1c3b7 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:04.647428 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e92673-419d-4ace-9a76-a494b01fccec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:04.648111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e92673-419d-4ace-9a76-a494b01fccec admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:04.648390 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e92673-419d-4ace-9a76-a494b01fccec admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:04.648668 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e92673-419d-4ace-9a76-a494b01fccec admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:04.649408 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e92673-419d-4ace-9a76-a494b01fccec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:04.651425 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e92673-419d-4ace-9a76-a494b01fccec admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:04.675186 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1287/2551] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:31:04 2022] POST /identity/v3/users => generated 307 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:04.679598 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d7504-300a-4c1d-b992-119c8a153800 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:04.690660 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d7504-300a-4c1d-b992-119c8a153800 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:04.695877 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e1f66-2507-4463-b872-1121823a6ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:04.696659 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e1f66-2507-4463-b872-1121823a6ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:04.697051 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e1f66-2507-4463-b872-1121823a6ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:04.697608 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e1f66-2507-4463-b872-1121823a6ea3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:04.698209 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e1f66-2507-4463-b872-1121823a6ea3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:04.699837 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e1f66-2507-4463-b872-1121823a6ea3 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:04.711690 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1265/2552] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:04.716290 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e92673-419d-4ace-9a76-a494b01fccec admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:04.727098 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e92673-419d-4ace-9a76-a494b01fccec admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:04.732710 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af9a69e-8611-4166-89f2-fc4af95a9396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:04.733731 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af9a69e-8611-4166-89f2-fc4af95a9396 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:04.734007 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af9a69e-8611-4166-89f2-fc4af95a9396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:04.734272 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af9a69e-8611-4166-89f2-fc4af95a9396 admin admin] PATH_INFO: `/v3/projects/81669794d56a4f7c9414e0aedba04085/users/48bd1cfede604364a1a404f403571928/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:04.749916 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af9a69e-8611-4166-89f2-fc4af95a9396 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81669794d56a4f7c9414e0aedba04085, user_id=48bd1cfede604364a1a404f403571928, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:04.752616 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af9a69e-8611-4166-89f2-fc4af95a9396 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:04.768250 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1288/2553] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:31:04 2022] PUT /identity/v3/projects/81669794d56a4f7c9414e0aedba04085/users/48bd1cfede604364a1a404f403571928/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:04.772449 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e1f66-2507-4463-b872-1121823a6ea3 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:04.782835 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e1f66-2507-4463-b872-1121823a6ea3 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:04.841170 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ddd29b-45d7-4170-8ce0-1b142403458a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:04.841844 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddd29b-45d7-4170-8ce0-1b142403458a admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:04.842119 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddd29b-45d7-4170-8ce0-1b142403458a admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:04.842396 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddd29b-45d7-4170-8ce0-1b142403458a admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:04.843120 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ddd29b-45d7-4170-8ce0-1b142403458a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:04.844725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ddd29b-45d7-4170-8ce0-1b142403458a admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:04.854846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1266/2554] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:04 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:04.859486 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af9a69e-8611-4166-89f2-fc4af95a9396 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:04.871361 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af9a69e-8611-4166-89f2-fc4af95a9396 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:04.877213 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56942d7b-383c-402d-941e-cc014c7a6870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:04.878179 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56942d7b-383c-402d-941e-cc014c7a6870 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:04.878440 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56942d7b-383c-402d-941e-cc014c7a6870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:04.878733 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56942d7b-383c-402d-941e-cc014c7a6870 admin admin] PATH_INFO: `/v3/projects/81669794d56a4f7c9414e0aedba04085/users/48bd1cfede604364a1a404f403571928/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:04.880809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56942d7b-383c-402d-941e-cc014c7a6870 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81669794d56a4f7c9414e0aedba04085, user_id=48bd1cfede604364a1a404f403571928, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:04.883864 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56942d7b-383c-402d-941e-cc014c7a6870 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:04.895319 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1289/2555] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:31:04 2022] PUT /identity/v3/projects/81669794d56a4f7c9414e0aedba04085/users/48bd1cfede604364a1a404f403571928/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:04.911302 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79583cfc-d9f6-4625-999a-13ed86f7dad5 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:04.911782 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79583cfc-d9f6-4625-999a-13ed86f7dad5 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:04.912145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79583cfc-d9f6-4625-999a-13ed86f7dad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:04.941959 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-79583cfc-d9f6-4625-999a-13ed86f7dad5 None None] MFA Rules not processed for user `48bd1cfede604364a1a404f403571928`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:04.943205 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-79583cfc-d9f6-4625-999a-13ed86f7dad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:05.015272 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1267/2556] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:04 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:05.031500 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56942d7b-383c-402d-941e-cc014c7a6870 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:05.033355 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-56942d7b-383c-402d-941e-cc014c7a6870 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:05.091898 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:05.095731 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56942d7b-383c-402d-941e-cc014c7a6870 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:05.100423 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad2063-7644-4f97-8389-7770bea83bad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:05.101106 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad2063-7644-4f97-8389-7770bea83bad service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:05.101390 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad2063-7644-4f97-8389-7770bea83bad service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:05.101664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad2063-7644-4f97-8389-7770bea83bad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:05.111673 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccad2063-7644-4f97-8389-7770bea83bad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:05.113181 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccad2063-7644-4f97-8389-7770bea83bad service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:05.123267 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1290/2557] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:05 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:05.214684 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79583cfc-d9f6-4625-999a-13ed86f7dad5 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:05.225081 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79583cfc-d9f6-4625-999a-13ed86f7dad5 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:05.229596 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e5261-8c06-450b-a499-3ca1de8107bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:05.230230 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e5261-8c06-450b-a499-3ca1de8107bd service cinder] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:05.230527 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e5261-8c06-450b-a499-3ca1de8107bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:05.230798 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e5261-8c06-450b-a499-3ca1de8107bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:05.305038 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:05.309451 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80e5261-8c06-450b-a499-3ca1de8107bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:05.311233 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80e5261-8c06-450b-a499-3ca1de8107bd service cinder] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:05.322315 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1268/2558] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:05 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:05.849118 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad2063-7644-4f97-8389-7770bea83bad service cinder] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:05.850632 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-ccad2063-7644-4f97-8389-7770bea83bad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:05.939368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad2063-7644-4f97-8389-7770bea83bad service cinder] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:05.943915 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4fb24-f166-49ee-a5ca-97e5d6bb9000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:05.944793 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4fb24-f166-49ee-a5ca-97e5d6bb9000 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:05.945112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4fb24-f166-49ee-a5ca-97e5d6bb9000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:05.945388 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4fb24-f166-49ee-a5ca-97e5d6bb9000 admin admin] PATH_INFO: `/v3/users/cb41863ac20146eba2ba178b6744d692` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:05.946595 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4fb24-f166-49ee-a5ca-97e5d6bb9000 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb41863ac20146eba2ba178b6744d692)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:05.948851 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d4fb24-f166-49ee-a5ca-97e5d6bb9000 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:06.007001 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-a3d4fb24-f166-49ee-a5ca-97e5d6bb9000 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb41863ac20146eba2ba178b6744d692'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:06.011056 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-a3d4fb24-f166-49ee-a5ca-97e5d6bb9000 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb41863ac20146eba2ba178b6744d692'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:06.020578 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-a3d4fb24-f166-49ee-a5ca-97e5d6bb9000 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb41863ac20146eba2ba178b6744d692'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:06.021392 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-a3d4fb24-f166-49ee-a5ca-97e5d6bb9000 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb41863ac20146eba2ba178b6744d692'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:06.035073 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-a3d4fb24-f166-49ee-a5ca-97e5d6bb9000 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb41863ac20146eba2ba178b6744d692'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:06.049079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1291/2559] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:31:05 2022] DELETE /identity/v3/users/cb41863ac20146eba2ba178b6744d692 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:06.053705 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7fbc9b-93bf-42a4-9901-a4cc9a5c1b54 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:06.054186 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7fbc9b-93bf-42a4-9901-a4cc9a5c1b54 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:06.054390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7fbc9b-93bf-42a4-9901-a4cc9a5c1b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:06.072647 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-3d7fbc9b-93bf-42a4-9901-a4cc9a5c1b54 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:06.074094 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-3d7fbc9b-93bf-42a4-9901-a4cc9a5c1b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:06.171807 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1269/2560] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:06.187751 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4fb24-f166-49ee-a5ca-97e5d6bb9000 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:06.190555 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-a3d4fb24-f166-49ee-a5ca-97e5d6bb9000 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:06.297785 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4fb24-f166-49ee-a5ca-97e5d6bb9000 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:06.301809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12152b11-0bee-4542-be33-d379e1545e18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:06.303269 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12152b11-0bee-4542-be33-d379e1545e18 service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:06.303653 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12152b11-0bee-4542-be33-d379e1545e18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:06.304312 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12152b11-0bee-4542-be33-d379e1545e18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:06.316549 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12152b11-0bee-4542-be33-d379e1545e18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:06.320199 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12152b11-0bee-4542-be33-d379e1545e18 service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:06.332107 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1292/2561] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:31:06.883934 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7fbc9b-93bf-42a4-9901-a4cc9a5c1b54 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:06.885792 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-3d7fbc9b-93bf-42a4-9901-a4cc9a5c1b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:06.909323 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7fbc9b-93bf-42a4-9901-a4cc9a5c1b54 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:06.920753 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc553c01-5183-459e-b259-bf8452b16aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:06.921920 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc553c01-5183-459e-b259-bf8452b16aec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:06.922087 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc553c01-5183-459e-b259-bf8452b16aec admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:06.922366 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc553c01-5183-459e-b259-bf8452b16aec admin admin] PATH_INFO: `/v3/projects/cebfe8ed3e584a20b70bcaada12a2479` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:06.930188 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc553c01-5183-459e-b259-bf8452b16aec admin admin] RBAC: Authorizing `identity:delete_project(project_id=cebfe8ed3e584a20b70bcaada12a2479)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:06.932510 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc553c01-5183-459e-b259-bf8452b16aec admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:06.997855 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-cc553c01-5183-459e-b259-bf8452b16aec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cebfe8ed3e584a20b70bcaada12a2479', 'request_id': 'req-cc553c01-5183-459e-b259-bf8452b16aec'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:07.005570 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-cc553c01-5183-459e-b259-bf8452b16aec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cebfe8ed3e584a20b70bcaada12a2479', 'request_id': 'req-cc553c01-5183-459e-b259-bf8452b16aec'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:07.012166 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-cc553c01-5183-459e-b259-bf8452b16aec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cebfe8ed3e584a20b70bcaada12a2479', 'request_id': 'req-cc553c01-5183-459e-b259-bf8452b16aec'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:07.031417 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-cc553c01-5183-459e-b259-bf8452b16aec admin admin] The token cache is being invalidate because project cebfe8ed3e584a20b70bcaada12a2479 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:31:07.031765 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-cc553c01-5183-459e-b259-bf8452b16aec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:07.032610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-cc553c01-5183-459e-b259-bf8452b16aec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:07.034707 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1270/2562] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:31:06 2022] DELETE /identity/v3/projects/cebfe8ed3e584a20b70bcaada12a2479 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:07.039046 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12152b11-0bee-4542-be33-d379e1545e18 service neutron] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:07.063820 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12152b11-0bee-4542-be33-d379e1545e18 service neutron] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:07.089984 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:07.202816 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6df13-6804-4dfc-b4aa-ef6de8a95597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:07.203868 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6df13-6804-4dfc-b4aa-ef6de8a95597 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:07.204253 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6df13-6804-4dfc-b4aa-ef6de8a95597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:07.204650 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b6df13-6804-4dfc-b4aa-ef6de8a95597 admin admin] PATH_INFO: `/v3/users/9d46a2bdff074a02bfe8d489711d0040` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:07.207391 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b6df13-6804-4dfc-b4aa-ef6de8a95597 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d46a2bdff074a02bfe8d489711d0040)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:07.209660 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b6df13-6804-4dfc-b4aa-ef6de8a95597 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:07.328037 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f5b6df13-6804-4dfc-b4aa-ef6de8a95597 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d46a2bdff074a02bfe8d489711d0040'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:07.328648 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f5b6df13-6804-4dfc-b4aa-ef6de8a95597 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d46a2bdff074a02bfe8d489711d0040'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:07.343481 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f5b6df13-6804-4dfc-b4aa-ef6de8a95597 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d46a2bdff074a02bfe8d489711d0040'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:07.344322 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f5b6df13-6804-4dfc-b4aa-ef6de8a95597 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d46a2bdff074a02bfe8d489711d0040'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:07.371010 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f5b6df13-6804-4dfc-b4aa-ef6de8a95597 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d46a2bdff074a02bfe8d489711d0040'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:07.380507 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1293/2563] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:31:07 2022] DELETE /identity/v3/users/9d46a2bdff074a02bfe8d489711d0040 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:07.721763 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc553c01-5183-459e-b259-bf8452b16aec admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:07.723244 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-cc553c01-5183-459e-b259-bf8452b16aec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:07.738149 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc553c01-5183-459e-b259-bf8452b16aec admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:07.807550 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33289e8-6075-45ba-a242-2615f14cb575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:07.808528 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33289e8-6075-45ba-a242-2615f14cb575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:07.808876 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33289e8-6075-45ba-a242-2615f14cb575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:07.809200 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33289e8-6075-45ba-a242-2615f14cb575 admin admin] PATH_INFO: `/v3/projects/a0d53d48748142b0af90b27ac6e47cd6` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:07.810543 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33289e8-6075-45ba-a242-2615f14cb575 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0d53d48748142b0af90b27ac6e47cd6)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:07.812481 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33289e8-6075-45ba-a242-2615f14cb575 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:07.843026 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:07.875767 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-f33289e8-6075-45ba-a242-2615f14cb575 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0d53d48748142b0af90b27ac6e47cd6', 'request_id': 'req-f33289e8-6075-45ba-a242-2615f14cb575'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:07.883391 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-f33289e8-6075-45ba-a242-2615f14cb575 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0d53d48748142b0af90b27ac6e47cd6', 'request_id': 'req-f33289e8-6075-45ba-a242-2615f14cb575'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:07.891726 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-f33289e8-6075-45ba-a242-2615f14cb575 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0d53d48748142b0af90b27ac6e47cd6', 'request_id': 'req-f33289e8-6075-45ba-a242-2615f14cb575'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:07.910427 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-f33289e8-6075-45ba-a242-2615f14cb575 admin admin] The token cache is being invalidate because project a0d53d48748142b0af90b27ac6e47cd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:31:07.910807 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-f33289e8-6075-45ba-a242-2615f14cb575 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:07.915655 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-f33289e8-6075-45ba-a242-2615f14cb575 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:07.918330 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1271/2564] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:31:07 2022] DELETE /identity/v3/projects/a0d53d48748142b0af90b27ac6e47cd6 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:07.922407 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6df13-6804-4dfc-b4aa-ef6de8a95597 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:07.923988 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-f5b6df13-6804-4dfc-b4aa-ef6de8a95597 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:07.953897 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b6df13-6804-4dfc-b4aa-ef6de8a95597 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:08.099209 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee9d0b7-c752-4c12-8af6-1c52c7a83dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:08.100100 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9d0b7-c752-4c12-8af6-1c52c7a83dda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:08.100444 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9d0b7-c752-4c12-8af6-1c52c7a83dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:08.100789 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9d0b7-c752-4c12-8af6-1c52c7a83dda admin admin] PATH_INFO: `/v3/users/28d9fa11ccae404990539ad5185b1295` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:08.102360 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee9d0b7-c752-4c12-8af6-1c52c7a83dda admin admin] RBAC: Authorizing `identity:delete_user(user_id=28d9fa11ccae404990539ad5185b1295)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:08.106273 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee9d0b7-c752-4c12-8af6-1c52c7a83dda admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:08.210887 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:08.234443 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3ee9d0b7-c752-4c12-8af6-1c52c7a83dda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28d9fa11ccae404990539ad5185b1295'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:08.236875 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3ee9d0b7-c752-4c12-8af6-1c52c7a83dda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28d9fa11ccae404990539ad5185b1295'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:08.248536 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3ee9d0b7-c752-4c12-8af6-1c52c7a83dda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28d9fa11ccae404990539ad5185b1295'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:08.249961 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3ee9d0b7-c752-4c12-8af6-1c52c7a83dda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28d9fa11ccae404990539ad5185b1295'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:08.263932 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3ee9d0b7-c752-4c12-8af6-1c52c7a83dda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28d9fa11ccae404990539ad5185b1295'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:08.290399 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1294/2565] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:31:07 2022] DELETE /identity/v3/users/28d9fa11ccae404990539ad5185b1295 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:31:08.722310 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33289e8-6075-45ba-a242-2615f14cb575 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:08.724584 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-f33289e8-6075-45ba-a242-2615f14cb575 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:08.749167 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33289e8-6075-45ba-a242-2615f14cb575 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:08.835316 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f24f7-720a-4993-97fb-10c6cb7aa696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:08.836207 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f24f7-720a-4993-97fb-10c6cb7aa696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:08.836486 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f24f7-720a-4993-97fb-10c6cb7aa696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:08.836762 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f24f7-720a-4993-97fb-10c6cb7aa696 admin admin] PATH_INFO: `/v3/projects/e0efcd38aae442aaa6da959c75411d75` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:08.838839 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931f24f7-720a-4993-97fb-10c6cb7aa696 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0efcd38aae442aaa6da959c75411d75)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:08.841669 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931f24f7-720a-4993-97fb-10c6cb7aa696 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:08.895897 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-931f24f7-720a-4993-97fb-10c6cb7aa696 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0efcd38aae442aaa6da959c75411d75', 'request_id': 'req-931f24f7-720a-4993-97fb-10c6cb7aa696'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:08.903473 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-931f24f7-720a-4993-97fb-10c6cb7aa696 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0efcd38aae442aaa6da959c75411d75', 'request_id': 'req-931f24f7-720a-4993-97fb-10c6cb7aa696'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:08.909953 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-931f24f7-720a-4993-97fb-10c6cb7aa696 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0efcd38aae442aaa6da959c75411d75', 'request_id': 'req-931f24f7-720a-4993-97fb-10c6cb7aa696'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:08.923747 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-931f24f7-720a-4993-97fb-10c6cb7aa696 admin admin] The token cache is being invalidate because project e0efcd38aae442aaa6da959c75411d75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:31:08.924054 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-931f24f7-720a-4993-97fb-10c6cb7aa696 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:08.924662 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-931f24f7-720a-4993-97fb-10c6cb7aa696 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:08.926271 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1272/2566] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:31:08 2022] DELETE /identity/v3/projects/e0efcd38aae442aaa6da959c75411d75 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:08.933476 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f86171c-457d-4069-a59f-042fbdf599f8 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:08.933761 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f86171c-457d-4069-a59f-042fbdf599f8 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:08.934041 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f86171c-457d-4069-a59f-042fbdf599f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:08.952535 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-7f86171c-457d-4069-a59f-042fbdf599f8 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:08.953764 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-7f86171c-457d-4069-a59f-042fbdf599f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:09.032034 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1295/2567] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:09.042563 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e39b5b-337b-412a-834d-b366b813d250 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:09.042849 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e39b5b-337b-412a-834d-b366b813d250 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:09.043128 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e39b5b-337b-412a-834d-b366b813d250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:09.066143 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-99e39b5b-337b-412a-834d-b366b813d250 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:09.067400 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-99e39b5b-337b-412a-834d-b366b813d250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:09.081174 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1273/2568] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:31:09.084053 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f86171c-457d-4069-a59f-042fbdf599f8 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:09.096117 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f86171c-457d-4069-a59f-042fbdf599f8 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:09.101044 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc038da-80f6-4e0b-b544-069f41a3bcc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:09.101900 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc038da-80f6-4e0b-b544-069f41a3bcc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:09.102271 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc038da-80f6-4e0b-b544-069f41a3bcc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:09.102651 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc038da-80f6-4e0b-b544-069f41a3bcc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:09.103694 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc038da-80f6-4e0b-b544-069f41a3bcc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:31:09.104087 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc038da-80f6-4e0b-b544-069f41a3bcc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:09.106528 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc038da-80f6-4e0b-b544-069f41a3bcc8 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:09.122467 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1296/2569] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:31:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:31:09.127253 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e39b5b-337b-412a-834d-b366b813d250 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:09.138222 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e39b5b-337b-412a-834d-b366b813d250 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:09.143030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae9a9e-3219-4101-ad5d-1e8eb55adb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:09.143790 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae9a9e-3219-4101-ad5d-1e8eb55adb8c admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:09.144133 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae9a9e-3219-4101-ad5d-1e8eb55adb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:09.144472 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae9a9e-3219-4101-ad5d-1e8eb55adb8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:09.145288 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfae9a9e-3219-4101-ad5d-1e8eb55adb8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:09.147199 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfae9a9e-3219-4101-ad5d-1e8eb55adb8c admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:09.179640 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1274/2570] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:31:09 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:09.183338 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc038da-80f6-4e0b-b544-069f41a3bcc8 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:09.194619 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc038da-80f6-4e0b-b544-069f41a3bcc8 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:09.267008 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05fd726-e920-40b5-a343-0010911e0b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:09.268155 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05fd726-e920-40b5-a343-0010911e0b80 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:09.268447 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05fd726-e920-40b5-a343-0010911e0b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:09.268875 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05fd726-e920-40b5-a343-0010911e0b80 admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:09.269934 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05fd726-e920-40b5-a343-0010911e0b80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:09.276811 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05fd726-e920-40b5-a343-0010911e0b80 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:09.302566 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1297/2571] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:31:09 2022] POST /identity/v3/users => generated 283 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:09.306976 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae9a9e-3219-4101-ad5d-1e8eb55adb8c admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:09.321025 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae9a9e-3219-4101-ad5d-1e8eb55adb8c admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:09.327803 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad07a72-cd4f-4802-9e2e-ab54d5cb8019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:09.329498 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad07a72-cd4f-4802-9e2e-ab54d5cb8019 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:09.330105 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad07a72-cd4f-4802-9e2e-ab54d5cb8019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:09.330640 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad07a72-cd4f-4802-9e2e-ab54d5cb8019 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:09.331795 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad07a72-cd4f-4802-9e2e-ab54d5cb8019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:09.334687 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad07a72-cd4f-4802-9e2e-ab54d5cb8019 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:09.341933 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:09.348746 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1275/2572] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:31:09.352761 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05fd726-e920-40b5-a343-0010911e0b80 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:09.367346 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05fd726-e920-40b5-a343-0010911e0b80 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:09.397101 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8753ac8-8bb6-4b24-8af6-9c6eeef5d646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:09.397870 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8753ac8-8bb6-4b24-8af6-9c6eeef5d646 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:09.398215 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8753ac8-8bb6-4b24-8af6-9c6eeef5d646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:09.398563 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8753ac8-8bb6-4b24-8af6-9c6eeef5d646 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:09.399357 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8753ac8-8bb6-4b24-8af6-9c6eeef5d646 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:09.401465 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8753ac8-8bb6-4b24-8af6-9c6eeef5d646 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:09.413945 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1298/2573] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:09 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:09.418449 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad07a72-cd4f-4802-9e2e-ab54d5cb8019 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:09.434613 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad07a72-cd4f-4802-9e2e-ab54d5cb8019 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:09.440637 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce88f6-f54b-4e5e-a53d-8436651bd542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:09.442495 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce88f6-f54b-4e5e-a53d-8436651bd542 admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:09.444443 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce88f6-f54b-4e5e-a53d-8436651bd542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:09.444443 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce88f6-f54b-4e5e-a53d-8436651bd542 admin admin] PATH_INFO: `/v3/projects/141475de7c9742fcb8c11fc56d620bb0/users/4d1433ea32cd4b809ff71c787e6a26e4/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:09.475742 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce88f6-f54b-4e5e-a53d-8436651bd542 admin admin] RBAC: Authorizing `identity:create_grant(project_id=141475de7c9742fcb8c11fc56d620bb0, user_id=4d1433ea32cd4b809ff71c787e6a26e4, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:09.479646 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce88f6-f54b-4e5e-a53d-8436651bd542 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:09.499157 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1276/2574] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:31:09 2022] PUT /identity/v3/projects/141475de7c9742fcb8c11fc56d620bb0/users/4d1433ea32cd4b809ff71c787e6a26e4/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:09.511427 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282f682-4cf8-4a08-885a-325e38dc4c73 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:09.511723 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282f682-4cf8-4a08-885a-325e38dc4c73 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:09.511979 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282f682-4cf8-4a08-885a-325e38dc4c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:09.548299 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-2282f682-4cf8-4a08-885a-325e38dc4c73 None None] MFA Rules not processed for user `4d1433ea32cd4b809ff71c787e6a26e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:09.549595 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-2282f682-4cf8-4a08-885a-325e38dc4c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:09.555534 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:09.620517 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1299/2575] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:09 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:09.632739 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce88f6-f54b-4e5e-a53d-8436651bd542 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:09.635013 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-4cce88f6-f54b-4e5e-a53d-8436651bd542 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:09.694105 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce88f6-f54b-4e5e-a53d-8436651bd542 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:09.697951 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef51b05-6d45-4644-878b-52cacc03ebaa service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b3c9c5c8d2e4376b1b70d974d4cb851', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:09.698606 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef51b05-6d45-4644-878b-52cacc03ebaa service glance] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:09.698880 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef51b05-6d45-4644-878b-52cacc03ebaa service glance] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:09.699151 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef51b05-6d45-4644-878b-52cacc03ebaa service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:09.708262 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef51b05-6d45-4644-878b-52cacc03ebaa service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:09.709845 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef51b05-6d45-4644-878b-52cacc03ebaa service glance] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:09.719941 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1277/2576] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:09 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:09.934896 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2282f682-4cf8-4a08-885a-325e38dc4c73 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:09.937406 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-2282f682-4cf8-4a08-885a-325e38dc4c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:10.011263 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2282f682-4cf8-4a08-885a-325e38dc4c73 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:10.016280 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e612bb-7dfd-4b3d-b8ad-b617cfb6a4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:10.017907 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e612bb-7dfd-4b3d-b8ad-b617cfb6a4bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:10.018568 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e612bb-7dfd-4b3d-b8ad-b617cfb6a4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:10.018804 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e612bb-7dfd-4b3d-b8ad-b617cfb6a4bb admin admin] PATH_INFO: `/v3/users/f2e4508850bd40deb267494189bb63dc` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:10.020137 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e612bb-7dfd-4b3d-b8ad-b617cfb6a4bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2e4508850bd40deb267494189bb63dc)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:10.022814 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e612bb-7dfd-4b3d-b8ad-b617cfb6a4bb admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:10.105451 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-80e612bb-7dfd-4b3d-b8ad-b617cfb6a4bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2e4508850bd40deb267494189bb63dc'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:10.106130 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-80e612bb-7dfd-4b3d-b8ad-b617cfb6a4bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2e4508850bd40deb267494189bb63dc'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:10.116085 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-80e612bb-7dfd-4b3d-b8ad-b617cfb6a4bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2e4508850bd40deb267494189bb63dc'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:10.116800 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-80e612bb-7dfd-4b3d-b8ad-b617cfb6a4bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2e4508850bd40deb267494189bb63dc'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:10.136353 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-80e612bb-7dfd-4b3d-b8ad-b617cfb6a4bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2e4508850bd40deb267494189bb63dc'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:10.151114 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1300/2577] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:31:09 2022] DELETE /identity/v3/users/f2e4508850bd40deb267494189bb63dc => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:31:10.155624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075e3fd-9a86-4828-b1a9-2ba67a165486 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:10.156103 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075e3fd-9a86-4828-b1a9-2ba67a165486 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:10.156228 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075e3fd-9a86-4828-b1a9-2ba67a165486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:10.175609 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-d075e3fd-9a86-4828-b1a9-2ba67a165486 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:10.177051 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-d075e3fd-9a86-4828-b1a9-2ba67a165486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:10.262347 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1278/2578] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:10.272708 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e612bb-7dfd-4b3d-b8ad-b617cfb6a4bb admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:10.276388 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-80e612bb-7dfd-4b3d-b8ad-b617cfb6a4bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:10.309693 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:10.338477 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e612bb-7dfd-4b3d-b8ad-b617cfb6a4bb admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:10.342098 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d1abe-5cd4-4fdf-87fc-6b1ead214ecf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:10.342755 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d1abe-5cd4-4fdf-87fc-6b1ead214ecf service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:10.343033 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d1abe-5cd4-4fdf-87fc-6b1ead214ecf service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:10.343304 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d1abe-5cd4-4fdf-87fc-6b1ead214ecf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:10.352827 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d1abe-5cd4-4fdf-87fc-6b1ead214ecf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:10.354964 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d1abe-5cd4-4fdf-87fc-6b1ead214ecf service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:10.364686 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1301/2579] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:10.841566 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075e3fd-9a86-4828-b1a9-2ba67a165486 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:10.843237 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-d075e3fd-9a86-4828-b1a9-2ba67a165486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:10.861255 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d1abe-5cd4-4fdf-87fc-6b1ead214ecf service neutron] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:10.863273 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-d62d1abe-5cd4-4fdf-87fc-6b1ead214ecf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:10.863376 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075e3fd-9a86-4828-b1a9-2ba67a165486 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:10.868319 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c92def-c3bb-4ec2-8b57-f9b74e1ba02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:10.869602 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c92def-c3bb-4ec2-8b57-f9b74e1ba02b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:10.869871 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c92def-c3bb-4ec2-8b57-f9b74e1ba02b admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:10.870254 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c92def-c3bb-4ec2-8b57-f9b74e1ba02b admin admin] PATH_INFO: `/v3/projects/b3bc8b78ca6848c6a1bbd707ecb32e74` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:10.871914 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c92def-c3bb-4ec2-8b57-f9b74e1ba02b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3bc8b78ca6848c6a1bbd707ecb32e74)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:10.873917 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c92def-c3bb-4ec2-8b57-f9b74e1ba02b admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:10.879014 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d1abe-5cd4-4fdf-87fc-6b1ead214ecf service neutron] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:10.898720 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9321e-9efc-4bd1-8629-5b62ed28d10b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:10.899390 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9321e-9efc-4bd1-8629-5b62ed28d10b service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:10.899744 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9321e-9efc-4bd1-8629-5b62ed28d10b service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:10.900068 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd9321e-9efc-4bd1-8629-5b62ed28d10b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:10.902095 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-2fd9321e-9efc-4bd1-8629-5b62ed28d10b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:10.932396 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-68c92def-c3bb-4ec2-8b57-f9b74e1ba02b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3bc8b78ca6848c6a1bbd707ecb32e74', 'request_id': 'req-68c92def-c3bb-4ec2-8b57-f9b74e1ba02b'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:10.940190 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-68c92def-c3bb-4ec2-8b57-f9b74e1ba02b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3bc8b78ca6848c6a1bbd707ecb32e74', 'request_id': 'req-68c92def-c3bb-4ec2-8b57-f9b74e1ba02b'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:10.945942 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-68c92def-c3bb-4ec2-8b57-f9b74e1ba02b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3bc8b78ca6848c6a1bbd707ecb32e74', 'request_id': 'req-68c92def-c3bb-4ec2-8b57-f9b74e1ba02b'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:10.960373 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-68c92def-c3bb-4ec2-8b57-f9b74e1ba02b admin admin] The token cache is being invalidate because project b3bc8b78ca6848c6a1bbd707ecb32e74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:31:10.960663 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-68c92def-c3bb-4ec2-8b57-f9b74e1ba02b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:10.961326 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-68c92def-c3bb-4ec2-8b57-f9b74e1ba02b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:10.963484 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1279/2580] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:31:10 2022] DELETE /identity/v3/projects/b3bc8b78ca6848c6a1bbd707ecb32e74 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:10.967725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c92def-c3bb-4ec2-8b57-f9b74e1ba02b admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:10.969164 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-68c92def-c3bb-4ec2-8b57-f9b74e1ba02b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:10.974214 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:10.979341 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd9321e-9efc-4bd1-8629-5b62ed28d10b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:10.980847 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd9321e-9efc-4bd1-8629-5b62ed28d10b service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:10.991469 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1302/2581] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:10 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:11.041028 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c92def-c3bb-4ec2-8b57-f9b74e1ba02b admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:11.045755 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed412d57-2779-42ff-bd0a-b842d166ad8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:11.046698 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed412d57-2779-42ff-bd0a-b842d166ad8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:11.046977 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed412d57-2779-42ff-bd0a-b842d166ad8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:11.047180 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed412d57-2779-42ff-bd0a-b842d166ad8f admin admin] PATH_INFO: `/v3/users/48bd1cfede604364a1a404f403571928` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:11.048409 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed412d57-2779-42ff-bd0a-b842d166ad8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=48bd1cfede604364a1a404f403571928)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:11.050395 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed412d57-2779-42ff-bd0a-b842d166ad8f admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:11.104455 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-ed412d57-2779-42ff-bd0a-b842d166ad8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48bd1cfede604364a1a404f403571928'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:11.117729 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-ed412d57-2779-42ff-bd0a-b842d166ad8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48bd1cfede604364a1a404f403571928'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:11.127806 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-ed412d57-2779-42ff-bd0a-b842d166ad8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48bd1cfede604364a1a404f403571928'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:11.137278 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-ed412d57-2779-42ff-bd0a-b842d166ad8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48bd1cfede604364a1a404f403571928'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:11.138185 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-ed412d57-2779-42ff-bd0a-b842d166ad8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48bd1cfede604364a1a404f403571928'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:11.145317 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1280/2582] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:31:10 2022] DELETE /identity/v3/users/48bd1cfede604364a1a404f403571928 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:11.618308 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9321e-9efc-4bd1-8629-5b62ed28d10b service neutron] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:11.634250 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd9321e-9efc-4bd1-8629-5b62ed28d10b service neutron] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:11.707895 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6b2862-ca79-40c9-b37c-25f8dabd39e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:11.708879 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b2862-ca79-40c9-b37c-25f8dabd39e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:11.709189 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b2862-ca79-40c9-b37c-25f8dabd39e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:11.709465 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b2862-ca79-40c9-b37c-25f8dabd39e6 admin admin] PATH_INFO: `/v3/projects/81669794d56a4f7c9414e0aedba04085` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:11.710664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6b2862-ca79-40c9-b37c-25f8dabd39e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81669794d56a4f7c9414e0aedba04085)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:11.712689 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6b2862-ca79-40c9-b37c-25f8dabd39e6 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:11.771256 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-0f6b2862-ca79-40c9-b37c-25f8dabd39e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81669794d56a4f7c9414e0aedba04085', 'request_id': 'req-0f6b2862-ca79-40c9-b37c-25f8dabd39e6'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:11.778437 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-0f6b2862-ca79-40c9-b37c-25f8dabd39e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81669794d56a4f7c9414e0aedba04085', 'request_id': 'req-0f6b2862-ca79-40c9-b37c-25f8dabd39e6'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:11.809551 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-0f6b2862-ca79-40c9-b37c-25f8dabd39e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81669794d56a4f7c9414e0aedba04085', 'request_id': 'req-0f6b2862-ca79-40c9-b37c-25f8dabd39e6'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:11.817125 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:11.825816 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-0f6b2862-ca79-40c9-b37c-25f8dabd39e6 admin admin] The token cache is being invalidate because project 81669794d56a4f7c9414e0aedba04085 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:31:11.826313 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-0f6b2862-ca79-40c9-b37c-25f8dabd39e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:11.828291 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-0f6b2862-ca79-40c9-b37c-25f8dabd39e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:11.835344 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1303/2583] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:31:11 2022] DELETE /identity/v3/projects/81669794d56a4f7c9414e0aedba04085 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:11.837044 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e67aa5-0051-4697-93f1-1a260a8092f1 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:11.837589 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e67aa5-0051-4697-93f1-1a260a8092f1 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:11.837693 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e67aa5-0051-4697-93f1-1a260a8092f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:11.878326 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-b9e67aa5-0051-4697-93f1-1a260a8092f1 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:11.879898 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-b9e67aa5-0051-4697-93f1-1a260a8092f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:11.880495 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6b2862-ca79-40c9-b37c-25f8dabd39e6 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:11.882692 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-0f6b2862-ca79-40c9-b37c-25f8dabd39e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:11.978390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1281/2584] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:11.989094 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac4925d-fb54-40c7-a795-a2d2b87ab637 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:11.989312 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6b2862-ca79-40c9-b37c-25f8dabd39e6 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:11.990306 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac4925d-fb54-40c7-a795-a2d2b87ab637 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:11.991243 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac4925d-fb54-40c7-a795-a2d2b87ab637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:11.997685 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f26ab-6611-4172-8bf8-eced080c1f8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:11.999025 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f26ab-6611-4172-8bf8-eced080c1f8a service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:11.999266 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f26ab-6611-4172-8bf8-eced080c1f8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:12.000133 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f26ab-6611-4172-8bf8-eced080c1f8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:12.007087 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-dd4f26ab-6611-4172-8bf8-eced080c1f8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:12.032731 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-5ac4925d-fb54-40c7-a795-a2d2b87ab637 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:12.034233 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-5ac4925d-fb54-40c7-a795-a2d2b87ab637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:12.035976 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4f26ab-6611-4172-8bf8-eced080c1f8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:12.037899 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4f26ab-6611-4172-8bf8-eced080c1f8a service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:12.061330 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1304/2585] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:31:12.063689 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1282/2586] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:12.064584 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f26ab-6611-4172-8bf8-eced080c1f8a service neutron] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:12.081827 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f26ab-6611-4172-8bf8-eced080c1f8a service neutron] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:12.087483 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90559ac-dbee-45d2-aa5b-ff881877f811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:12.088192 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90559ac-dbee-45d2-aa5b-ff881877f811 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:12.088759 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90559ac-dbee-45d2-aa5b-ff881877f811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:12.089161 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90559ac-dbee-45d2-aa5b-ff881877f811 admin admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:12.089990 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90559ac-dbee-45d2-aa5b-ff881877f811 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:31:12.090357 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90559ac-dbee-45d2-aa5b-ff881877f811 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:12.092175 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90559ac-dbee-45d2-aa5b-ff881877f811 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:12.109416 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1305/2587] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:31:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:31:12.113343 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac4925d-fb54-40c7-a795-a2d2b87ab637 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:12.123674 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac4925d-fb54-40c7-a795-a2d2b87ab637 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:12.129368 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afdf159-e7b8-44d4-98ec-f93e34f53f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:12.130049 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afdf159-e7b8-44d4-98ec-f93e34f53f75 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:12.130520 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afdf159-e7b8-44d4-98ec-f93e34f53f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:12.132297 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afdf159-e7b8-44d4-98ec-f93e34f53f75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:12.132601 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afdf159-e7b8-44d4-98ec-f93e34f53f75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:12.136449 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afdf159-e7b8-44d4-98ec-f93e34f53f75 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:12.161541 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:12.181059 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1283/2588] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:31:12 2022] POST /identity/v3/projects => generated 381 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:31:12.184330 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90559ac-dbee-45d2-aa5b-ff881877f811 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:12.198211 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90559ac-dbee-45d2-aa5b-ff881877f811 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:12.277088 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a4ab4-875f-467e-a702-dc5b5dd5b0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:12.277792 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a4ab4-875f-467e-a702-dc5b5dd5b0a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:12.278078 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a4ab4-875f-467e-a702-dc5b5dd5b0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:12.278355 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a4ab4-875f-467e-a702-dc5b5dd5b0a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:12.279054 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429a4ab4-875f-467e-a702-dc5b5dd5b0a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:12.281118 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429a4ab4-875f-467e-a702-dc5b5dd5b0a6 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:12.328614 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1306/2589] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:31:12 2022] POST /identity/v3/users => generated 291 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:12.328969 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afdf159-e7b8-44d4-98ec-f93e34f53f75 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:12.346532 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afdf159-e7b8-44d4-98ec-f93e34f53f75 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:12.359958 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e788fd3-14c6-40fc-abec-58c29abd7852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:12.361492 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e788fd3-14c6-40fc-abec-58c29abd7852 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:12.361849 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e788fd3-14c6-40fc-abec-58c29abd7852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:12.362407 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e788fd3-14c6-40fc-abec-58c29abd7852 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:12.363068 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e788fd3-14c6-40fc-abec-58c29abd7852 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:12.364977 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e788fd3-14c6-40fc-abec-58c29abd7852 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:12.382618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1284/2590] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:31:12.386506 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a4ab4-875f-467e-a702-dc5b5dd5b0a6 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:12.401537 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a4ab4-875f-467e-a702-dc5b5dd5b0a6 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:12.407962 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c6ca59-2532-4a33-8c58-29b8145da36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:12.409042 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6ca59-2532-4a33-8c58-29b8145da36e admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:12.409429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6ca59-2532-4a33-8c58-29b8145da36e admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:12.417925 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6ca59-2532-4a33-8c58-29b8145da36e admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:12.417925 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c6ca59-2532-4a33-8c58-29b8145da36e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:12.417925 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c6ca59-2532-4a33-8c58-29b8145da36e admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:12.429013 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1307/2591] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:12.433051 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e788fd3-14c6-40fc-abec-58c29abd7852 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:12.454140 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e788fd3-14c6-40fc-abec-58c29abd7852 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:12.460210 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a6a66-e6cc-4045-bcaa-e8be00f7077e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:12.461339 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a6a66-e6cc-4045-bcaa-e8be00f7077e admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:12.461745 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a6a66-e6cc-4045-bcaa-e8be00f7077e admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:12.462120 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a6a66-e6cc-4045-bcaa-e8be00f7077e admin admin] PATH_INFO: `/v3/projects/ef7868209d6d4fb69a483fff24d2af19/users/72c5e6acbf1b46e6abec65e410ae89b5/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:12.492254 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233a6a66-e6cc-4045-bcaa-e8be00f7077e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef7868209d6d4fb69a483fff24d2af19, user_id=72c5e6acbf1b46e6abec65e410ae89b5, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:12.495106 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233a6a66-e6cc-4045-bcaa-e8be00f7077e admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:12.524889 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1285/2592] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:31:12 2022] PUT /identity/v3/projects/ef7868209d6d4fb69a483fff24d2af19/users/72c5e6acbf1b46e6abec65e410ae89b5/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:12.535596 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe6d6b-21aa-4d4d-a452-c21296da5f1e None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:12.536175 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe6d6b-21aa-4d4d-a452-c21296da5f1e None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:12.537091 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe6d6b-21aa-4d4d-a452-c21296da5f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:12.590077 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-87fe6d6b-21aa-4d4d-a452-c21296da5f1e None None] MFA Rules not processed for user `72c5e6acbf1b46e6abec65e410ae89b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:12.591762 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-87fe6d6b-21aa-4d4d-a452-c21296da5f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:12.692630 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:12.730582 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1308/2593] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:12.732159 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a6a66-e6cc-4045-bcaa-e8be00f7077e admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:12.825355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a6a66-e6cc-4045-bcaa-e8be00f7077e admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:12.834928 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bddfd7-de33-4a48-b7a5-145ac4456a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:12.835804 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bddfd7-de33-4a48-b7a5-145ac4456a87 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:12.836234 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bddfd7-de33-4a48-b7a5-145ac4456a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:12.836513 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bddfd7-de33-4a48-b7a5-145ac4456a87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:12.837592 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bddfd7-de33-4a48-b7a5-145ac4456a87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:12.839939 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bddfd7-de33-4a48-b7a5-145ac4456a87 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:12.867301 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1286/2594] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:31:12 2022] POST /identity/v3/projects => generated 383 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:12.872322 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe6d6b-21aa-4d4d-a452-c21296da5f1e None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:12.887763 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe6d6b-21aa-4d4d-a452-c21296da5f1e None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:12.955326 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7e24e-89b3-4021-910e-c35775aec2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:12.956301 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7e24e-89b3-4021-910e-c35775aec2d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:12.956778 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7e24e-89b3-4021-910e-c35775aec2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:12.957242 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7e24e-89b3-4021-910e-c35775aec2d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:12.958110 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a7e24e-89b3-4021-910e-c35775aec2d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:12.960170 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a7e24e-89b3-4021-910e-c35775aec2d0 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:12.986982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1309/2595] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:31:12 2022] POST /identity/v3/users => generated 298 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:31:12.991249 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bddfd7-de33-4a48-b7a5-145ac4456a87 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:13.002850 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bddfd7-de33-4a48-b7a5-145ac4456a87 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:13.008229 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820a95f-999a-495f-a2cb-ca266d948a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:13.008902 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820a95f-999a-495f-a2cb-ca266d948a8f admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:13.009368 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820a95f-999a-495f-a2cb-ca266d948a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:13.009672 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820a95f-999a-495f-a2cb-ca266d948a8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:13.010710 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9820a95f-999a-495f-a2cb-ca266d948a8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:13.011990 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9820a95f-999a-495f-a2cb-ca266d948a8f admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:13.026861 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1287/2596] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:13.031785 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7e24e-89b3-4021-910e-c35775aec2d0 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:13.044945 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7e24e-89b3-4021-910e-c35775aec2d0 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:13.053086 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a7a99-d2ff-4d25-980c-1f6684441f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:13.054781 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a7a99-d2ff-4d25-980c-1f6684441f77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:13.055066 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a7a99-d2ff-4d25-980c-1f6684441f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:13.055348 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a7a99-d2ff-4d25-980c-1f6684441f77 admin admin] PATH_INFO: `/v3/projects/f68ca1558b724506b3542f8cfc917e6e/users/856c52bdabef473abb6699509dcb6a7c/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:13.081739 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1a7a99-d2ff-4d25-980c-1f6684441f77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f68ca1558b724506b3542f8cfc917e6e, user_id=856c52bdabef473abb6699509dcb6a7c, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:13.087584 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1a7a99-d2ff-4d25-980c-1f6684441f77 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:13.105545 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1310/2597] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:31:13 2022] PUT /identity/v3/projects/f68ca1558b724506b3542f8cfc917e6e/users/856c52bdabef473abb6699509dcb6a7c/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:13.109618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820a95f-999a-495f-a2cb-ca266d948a8f admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:13.128268 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820a95f-999a-495f-a2cb-ca266d948a8f admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:13.245597 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ab811a-6214-4a6c-8c0d-ca04a6fb915c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:13.246305 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab811a-6214-4a6c-8c0d-ca04a6fb915c admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:13.246584 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab811a-6214-4a6c-8c0d-ca04a6fb915c admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:13.246854 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab811a-6214-4a6c-8c0d-ca04a6fb915c admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:13.247629 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ab811a-6214-4a6c-8c0d-ca04a6fb915c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:13.249460 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ab811a-6214-4a6c-8c0d-ca04a6fb915c admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:13.265993 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1288/2598] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:13 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:13.274725 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a7a99-d2ff-4d25-980c-1f6684441f77 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:13.292323 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a7a99-d2ff-4d25-980c-1f6684441f77 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:13.297381 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed50945-8e40-4ffb-b0bc-b633b9b9124a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:13.298562 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed50945-8e40-4ffb-b0bc-b633b9b9124a admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:13.298866 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed50945-8e40-4ffb-b0bc-b633b9b9124a admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:13.299144 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed50945-8e40-4ffb-b0bc-b633b9b9124a admin admin] PATH_INFO: `/v3/projects/f68ca1558b724506b3542f8cfc917e6e/users/856c52bdabef473abb6699509dcb6a7c/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:13.301670 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed50945-8e40-4ffb-b0bc-b633b9b9124a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f68ca1558b724506b3542f8cfc917e6e, user_id=856c52bdabef473abb6699509dcb6a7c, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:13.305376 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed50945-8e40-4ffb-b0bc-b633b9b9124a admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:13.319156 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1311/2599] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:31:13 2022] PUT /identity/v3/projects/f68ca1558b724506b3542f8cfc917e6e/users/856c52bdabef473abb6699509dcb6a7c/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:13.331974 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024da69-9575-4ea1-aa59-97e4e707521f None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:13.332182 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024da69-9575-4ea1-aa59-97e4e707521f None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:13.333083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024da69-9575-4ea1-aa59-97e4e707521f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:13.385566 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-9024da69-9575-4ea1-aa59-97e4e707521f None None] MFA Rules not processed for user `856c52bdabef473abb6699509dcb6a7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:13.387168 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-9024da69-9575-4ea1-aa59-97e4e707521f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:13.413766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:13.482874 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1289/2600] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:13 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:13.496990 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed50945-8e40-4ffb-b0bc-b633b9b9124a admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:13.499180 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-3ed50945-8e40-4ffb-b0bc-b633b9b9124a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:13.591298 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed50945-8e40-4ffb-b0bc-b633b9b9124a admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:13.595353 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e3440f-91bf-4f37-ad42-7969c4634126 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:13.596175 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e3440f-91bf-4f37-ad42-7969c4634126 service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:13.596501 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e3440f-91bf-4f37-ad42-7969c4634126 service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:13.596871 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e3440f-91bf-4f37-ad42-7969c4634126 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:13.608633 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e3440f-91bf-4f37-ad42-7969c4634126 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:13.610604 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e3440f-91bf-4f37-ad42-7969c4634126 service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:13.623735 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1312/2601] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:13 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:13.749920 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9024da69-9575-4ea1-aa59-97e4e707521f None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:13.766513 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9024da69-9575-4ea1-aa59-97e4e707521f None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:13.770554 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd965215-6032-42f5-9b6f-b0c3f049d18d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:13.771332 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd965215-6032-42f5-9b6f-b0c3f049d18d service cinder] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:13.771716 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd965215-6032-42f5-9b6f-b0c3f049d18d service cinder] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:13.772095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd965215-6032-42f5-9b6f-b0c3f049d18d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:13.844067 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd965215-6032-42f5-9b6f-b0c3f049d18d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:13.845705 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd965215-6032-42f5-9b6f-b0c3f049d18d service cinder] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:13.858573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1290/2602] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:16.728876 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e3440f-91bf-4f37-ad42-7969c4634126 service cinder] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:16.730771 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-f2e3440f-91bf-4f37-ad42-7969c4634126 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:16.851374 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:16.875484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e3440f-91bf-4f37-ad42-7969c4634126 service cinder] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:16.884665 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d00b3d-6f9a-4279-b7ba-9e14fb2aea8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:16.884665 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d00b3d-6f9a-4279-b7ba-9e14fb2aea8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:16.884665 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d00b3d-6f9a-4279-b7ba-9e14fb2aea8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:16.884665 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d00b3d-6f9a-4279-b7ba-9e14fb2aea8a admin admin] PATH_INFO: `/v3/users/dd9938d83dc84ae494310741e2cba2ec` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:16.886487 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d00b3d-6f9a-4279-b7ba-9e14fb2aea8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd9938d83dc84ae494310741e2cba2ec)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:16.888903 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d00b3d-6f9a-4279-b7ba-9e14fb2aea8a admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:16.960157 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-62d00b3d-6f9a-4279-b7ba-9e14fb2aea8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd9938d83dc84ae494310741e2cba2ec'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:16.961034 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-62d00b3d-6f9a-4279-b7ba-9e14fb2aea8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd9938d83dc84ae494310741e2cba2ec'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:16.973663 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-62d00b3d-6f9a-4279-b7ba-9e14fb2aea8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd9938d83dc84ae494310741e2cba2ec'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:16.974300 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-62d00b3d-6f9a-4279-b7ba-9e14fb2aea8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd9938d83dc84ae494310741e2cba2ec'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:16.987983 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-62d00b3d-6f9a-4279-b7ba-9e14fb2aea8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd9938d83dc84ae494310741e2cba2ec'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:17.001684 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1313/2603] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:31:16 2022] DELETE /identity/v3/users/dd9938d83dc84ae494310741e2cba2ec => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:17.198030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd965215-6032-42f5-9b6f-b0c3f049d18d service cinder] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:17.209959 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd965215-6032-42f5-9b6f-b0c3f049d18d service cinder] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:17.272189 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f303e-3d9b-4e8a-a958-0281b71063ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:17.272984 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f303e-3d9b-4e8a-a958-0281b71063ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:17.273288 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f303e-3d9b-4e8a-a958-0281b71063ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:17.273556 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f303e-3d9b-4e8a-a958-0281b71063ff admin admin] PATH_INFO: `/v3/projects/7e2e4b63d6af43058e0dc76d8ceac844` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:17.274940 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042f303e-3d9b-4e8a-a958-0281b71063ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e2e4b63d6af43058e0dc76d8ceac844)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:17.276912 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042f303e-3d9b-4e8a-a958-0281b71063ff admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:17.327810 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-042f303e-3d9b-4e8a-a958-0281b71063ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e2e4b63d6af43058e0dc76d8ceac844', 'request_id': 'req-042f303e-3d9b-4e8a-a958-0281b71063ff'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:17.334765 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-042f303e-3d9b-4e8a-a958-0281b71063ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e2e4b63d6af43058e0dc76d8ceac844', 'request_id': 'req-042f303e-3d9b-4e8a-a958-0281b71063ff'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:17.336011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:17.342022 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-042f303e-3d9b-4e8a-a958-0281b71063ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e2e4b63d6af43058e0dc76d8ceac844', 'request_id': 'req-042f303e-3d9b-4e8a-a958-0281b71063ff'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:17.358609 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-042f303e-3d9b-4e8a-a958-0281b71063ff admin admin] The token cache is being invalidate because project 7e2e4b63d6af43058e0dc76d8ceac844 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:31:17.358919 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-042f303e-3d9b-4e8a-a958-0281b71063ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:17.359509 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-042f303e-3d9b-4e8a-a958-0281b71063ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:17.361113 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1291/2604] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:31:17 2022] DELETE /identity/v3/projects/7e2e4b63d6af43058e0dc76d8ceac844 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:17.366288 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef653b58-4816-4073-b5ce-4587611f0b12 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:17.366582 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef653b58-4816-4073-b5ce-4587611f0b12 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:17.367100 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef653b58-4816-4073-b5ce-4587611f0b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:17.387360 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-ef653b58-4816-4073-b5ce-4587611f0b12 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:17.388582 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-ef653b58-4816-4073-b5ce-4587611f0b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:17.457333 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1314/2605] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:17.466533 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ad853-7d53-407a-90df-94f48e125283 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:17.466820 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ad853-7d53-407a-90df-94f48e125283 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:17.467093 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1ad853-7d53-407a-90df-94f48e125283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:17.487439 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-2a1ad853-7d53-407a-90df-94f48e125283 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:17.488671 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-2a1ad853-7d53-407a-90df-94f48e125283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:17.498152 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1292/2606] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:17.503788 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef653b58-4816-4073-b5ce-4587611f0b12 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:17.514872 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef653b58-4816-4073-b5ce-4587611f0b12 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:17.520158 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da8e466-aef5-4454-a636-cbc454b5c90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:17.520803 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8e466-aef5-4454-a636-cbc454b5c90e admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:17.521147 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8e466-aef5-4454-a636-cbc454b5c90e admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:17.521451 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da8e466-aef5-4454-a636-cbc454b5c90e admin admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:17.522180 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da8e466-aef5-4454-a636-cbc454b5c90e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:31:17.522551 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da8e466-aef5-4454-a636-cbc454b5c90e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:17.524146 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da8e466-aef5-4454-a636-cbc454b5c90e admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:17.536623 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1315/2607] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:31:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:31:17.542018 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1ad853-7d53-407a-90df-94f48e125283 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:17.551883 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1ad853-7d53-407a-90df-94f48e125283 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:17.557724 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87951b8-f6bb-4bb5-a1a6-184941b3b369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:17.558386 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87951b8-f6bb-4bb5-a1a6-184941b3b369 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:17.558665 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87951b8-f6bb-4bb5-a1a6-184941b3b369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:17.558938 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87951b8-f6bb-4bb5-a1a6-184941b3b369 admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:17.559652 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87951b8-f6bb-4bb5-a1a6-184941b3b369 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:17.561632 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87951b8-f6bb-4bb5-a1a6-184941b3b369 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:17.598348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1293/2608] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:31:17 2022] POST /identity/v3/projects => generated 363 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:17.604420 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da8e466-aef5-4454-a636-cbc454b5c90e admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:17.615310 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da8e466-aef5-4454-a636-cbc454b5c90e admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:17.675676 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:17.687433 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6477ecbe-cbdb-423c-be27-ba8ad293e9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:17.688272 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477ecbe-cbdb-423c-be27-ba8ad293e9dc admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:17.688553 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477ecbe-cbdb-423c-be27-ba8ad293e9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:17.688832 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477ecbe-cbdb-423c-be27-ba8ad293e9dc admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:17.689559 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6477ecbe-cbdb-423c-be27-ba8ad293e9dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:17.691518 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6477ecbe-cbdb-423c-be27-ba8ad293e9dc admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:17.714613 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1316/2609] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:31:17 2022] POST /identity/v3/users => generated 282 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:17.719622 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87951b8-f6bb-4bb5-a1a6-184941b3b369 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:17.731123 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87951b8-f6bb-4bb5-a1a6-184941b3b369 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:17.738565 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87592d8-c582-47ef-9715-033015027978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:17.739244 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87592d8-c582-47ef-9715-033015027978 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:17.739526 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87592d8-c582-47ef-9715-033015027978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:17.739801 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87592d8-c582-47ef-9715-033015027978 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:17.740567 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87592d8-c582-47ef-9715-033015027978 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:17.742207 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87592d8-c582-47ef-9715-033015027978 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:17.753522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1294/2610] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:17.757901 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6477ecbe-cbdb-423c-be27-ba8ad293e9dc admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:17.773164 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6477ecbe-cbdb-423c-be27-ba8ad293e9dc admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:17.778102 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53faa2-f701-4f19-a62a-6b5b75c2bbc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:17.778784 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53faa2-f701-4f19-a62a-6b5b75c2bbc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:17.779055 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53faa2-f701-4f19-a62a-6b5b75c2bbc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:17.779322 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53faa2-f701-4f19-a62a-6b5b75c2bbc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:17.780070 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb53faa2-f701-4f19-a62a-6b5b75c2bbc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:17.781727 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb53faa2-f701-4f19-a62a-6b5b75c2bbc9 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:17.801177 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1317/2611] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:17.805160 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87592d8-c582-47ef-9715-033015027978 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:17.815722 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87592d8-c582-47ef-9715-033015027978 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:17.820222 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbfece2-847a-4f56-9379-702312ac822c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:17.821156 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbfece2-847a-4f56-9379-702312ac822c admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:17.821444 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbfece2-847a-4f56-9379-702312ac822c admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:17.821720 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbfece2-847a-4f56-9379-702312ac822c admin admin] PATH_INFO: `/v3/projects/b03f25d58f5b4ec58f91134036777cfe/users/9f096c73af1f420e87cb3e4de1e8e8b9/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:17.837548 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbfece2-847a-4f56-9379-702312ac822c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b03f25d58f5b4ec58f91134036777cfe, user_id=9f096c73af1f420e87cb3e4de1e8e8b9, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:17.840418 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbfece2-847a-4f56-9379-702312ac822c admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:17.855632 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1295/2612] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:31:17 2022] PUT /identity/v3/projects/b03f25d58f5b4ec58f91134036777cfe/users/9f096c73af1f420e87cb3e4de1e8e8b9/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:17.865873 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d38d1-78e8-4bc9-b297-ff2865740207 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:17.866161 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d38d1-78e8-4bc9-b297-ff2865740207 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:17.866438 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2d38d1-78e8-4bc9-b297-ff2865740207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:17.893468 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-5b2d38d1-78e8-4bc9-b297-ff2865740207 None None] MFA Rules not processed for user `9f096c73af1f420e87cb3e4de1e8e8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:17.894678 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-5b2d38d1-78e8-4bc9-b297-ff2865740207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:17.969488 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1318/2613] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:17 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:31:17.972363 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbfece2-847a-4f56-9379-702312ac822c admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:18.044020 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbfece2-847a-4f56-9379-702312ac822c admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:18.048786 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae854f06-917f-47fe-984a-661967ca5eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:18.049534 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae854f06-917f-47fe-984a-661967ca5eb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:18.049840 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae854f06-917f-47fe-984a-661967ca5eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:18.050140 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae854f06-917f-47fe-984a-661967ca5eb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:18.050919 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae854f06-917f-47fe-984a-661967ca5eb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:18.052945 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae854f06-917f-47fe-984a-661967ca5eb1 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:18.082376 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1296/2614] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:31:17 2022] POST /identity/v3/projects => generated 365 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:18.086519 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d38d1-78e8-4bc9-b297-ff2865740207 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:18.097819 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2d38d1-78e8-4bc9-b297-ff2865740207 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:18.160916 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa477c7-1208-4e60-89f1-b98ae473706b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:18.161622 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa477c7-1208-4e60-89f1-b98ae473706b admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:18.161919 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa477c7-1208-4e60-89f1-b98ae473706b admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:18.162188 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa477c7-1208-4e60-89f1-b98ae473706b admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:18.162885 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa477c7-1208-4e60-89f1-b98ae473706b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:18.164852 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa477c7-1208-4e60-89f1-b98ae473706b admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:18.187543 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1319/2615] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:31:18 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:31:18.191912 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae854f06-917f-47fe-984a-661967ca5eb1 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:18.204245 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae854f06-917f-47fe-984a-661967ca5eb1 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:18.209512 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1679a33-1780-4d58-9973-cd9750a9510b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:18.210173 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1679a33-1780-4d58-9973-cd9750a9510b admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:18.210448 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1679a33-1780-4d58-9973-cd9750a9510b admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:18.210725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1679a33-1780-4d58-9973-cd9750a9510b admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:18.211447 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1679a33-1780-4d58-9973-cd9750a9510b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:18.213201 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1679a33-1780-4d58-9973-cd9750a9510b admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:18.223857 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1297/2616] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:18.228297 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa477c7-1208-4e60-89f1-b98ae473706b admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:18.239843 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa477c7-1208-4e60-89f1-b98ae473706b admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:18.246292 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db599660-ea15-487d-84c8-963f46168d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:18.247187 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db599660-ea15-487d-84c8-963f46168d99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:18.247468 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db599660-ea15-487d-84c8-963f46168d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:18.247746 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db599660-ea15-487d-84c8-963f46168d99 admin admin] PATH_INFO: `/v3/projects/8afcf689df434505b5c327d610e7a53f/users/8605e7e981e8497d833b46c0394c19b3/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:18.266315 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db599660-ea15-487d-84c8-963f46168d99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8afcf689df434505b5c327d610e7a53f, user_id=8605e7e981e8497d833b46c0394c19b3, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:18.269097 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db599660-ea15-487d-84c8-963f46168d99 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:18.284522 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1320/2617] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:31:18 2022] PUT /identity/v3/projects/8afcf689df434505b5c327d610e7a53f/users/8605e7e981e8497d833b46c0394c19b3/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:18.288559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1679a33-1780-4d58-9973-cd9750a9510b admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:18.299268 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1679a33-1780-4d58-9973-cd9750a9510b admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:18.354408 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:18.364881 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b0eff2-4953-40de-b7ce-b6e7e6ce6c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:18.365831 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0eff2-4953-40de-b7ce-b6e7e6ce6c1d admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:18.366174 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0eff2-4953-40de-b7ce-b6e7e6ce6c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:18.366504 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0eff2-4953-40de-b7ce-b6e7e6ce6c1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:18.367310 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b0eff2-4953-40de-b7ce-b6e7e6ce6c1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:18.369592 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b0eff2-4953-40de-b7ce-b6e7e6ce6c1d admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:18.381280 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1298/2618] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:18.386200 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db599660-ea15-487d-84c8-963f46168d99 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:18.397125 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db599660-ea15-487d-84c8-963f46168d99 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:18.401971 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761719d-69ea-43f6-af00-e133a3c62fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:18.402905 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761719d-69ea-43f6-af00-e133a3c62fa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:18.403191 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761719d-69ea-43f6-af00-e133a3c62fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:18.403453 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761719d-69ea-43f6-af00-e133a3c62fa3 admin admin] PATH_INFO: `/v3/projects/8afcf689df434505b5c327d610e7a53f/users/8605e7e981e8497d833b46c0394c19b3/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:18.405419 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f761719d-69ea-43f6-af00-e133a3c62fa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8afcf689df434505b5c327d610e7a53f, user_id=8605e7e981e8497d833b46c0394c19b3, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:18.408052 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f761719d-69ea-43f6-af00-e133a3c62fa3 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:18.419296 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1321/2619] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:31:18 2022] PUT /identity/v3/projects/8afcf689df434505b5c327d610e7a53f/users/8605e7e981e8497d833b46c0394c19b3/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:18.430209 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587d060-fd83-4993-8b8e-140b8d347159 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:18.430491 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587d060-fd83-4993-8b8e-140b8d347159 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:18.430764 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1587d060-fd83-4993-8b8e-140b8d347159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:18.458638 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-1587d060-fd83-4993-8b8e-140b8d347159 None None] MFA Rules not processed for user `8605e7e981e8497d833b46c0394c19b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:18.460765 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-1587d060-fd83-4993-8b8e-140b8d347159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:18.532015 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1299/2620] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:18 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:18.543927 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761719d-69ea-43f6-af00-e133a3c62fa3 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:18.545828 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-f761719d-69ea-43f6-af00-e133a3c62fa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:18.560926 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:18.608349 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761719d-69ea-43f6-af00-e133a3c62fa3 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:18.612449 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43408906-f35f-4539-8edb-f6a387f089e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:18.613128 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43408906-f35f-4539-8edb-f6a387f089e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:18.613409 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43408906-f35f-4539-8edb-f6a387f089e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:18.613687 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43408906-f35f-4539-8edb-f6a387f089e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:18.623256 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43408906-f35f-4539-8edb-f6a387f089e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:18.624710 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43408906-f35f-4539-8edb-f6a387f089e0 service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:18.635467 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1322/2621] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:18 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:18.715211 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587d060-fd83-4993-8b8e-140b8d347159 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:18.725175 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1587d060-fd83-4993-8b8e-140b8d347159 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:18.729141 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6330db-b653-40fb-ab13-9041e41d49f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:18.729765 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6330db-b653-40fb-ab13-9041e41d49f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:18.730046 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6330db-b653-40fb-ab13-9041e41d49f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:18.730321 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6330db-b653-40fb-ab13-9041e41d49f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:18.788936 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6330db-b653-40fb-ab13-9041e41d49f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:18.790431 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6330db-b653-40fb-ab13-9041e41d49f8 service cinder] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:18.799683 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1300/2622] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:18 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:19.360980 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854efaac-74ab-4e0b-9584-e69f5931c51b None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:19.361502 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854efaac-74ab-4e0b-9584-e69f5931c51b None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:19.361609 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854efaac-74ab-4e0b-9584-e69f5931c51b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:19.383889 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-854efaac-74ab-4e0b-9584-e69f5931c51b None None] MFA Rules not processed for user `86a18e313e4944748aef58d54bfe7b13`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:19.385296 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-854efaac-74ab-4e0b-9584-e69f5931c51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:19.446737 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1323/2623] 10.222.0.50 () {58 vars in 1009 bytes} [Fri May 27 03:31:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:19.472552 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6330db-b653-40fb-ab13-9041e41d49f8 service cinder] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:19.474136 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-4e6330db-b653-40fb-ab13-9041e41d49f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:19.552875 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6330db-b653-40fb-ab13-9041e41d49f8 service cinder] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:19.558575 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c19042-b409-4d92-ace8-221a6b689cde service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de53e2f0fba4e0fac70887067c1963d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:19.559233 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c19042-b409-4d92-ace8-221a6b689cde service swift] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:19.559522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c19042-b409-4d92-ace8-221a6b689cde service swift] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:19.559798 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c19042-b409-4d92-ace8-221a6b689cde service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:19.574118 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c19042-b409-4d92-ace8-221a6b689cde service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:19.575886 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c19042-b409-4d92-ace8-221a6b689cde service swift] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:19.585395 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1301/2624] 10.222.0.50 () {60 vars in 1384 bytes} [Fri May 27 03:31:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:31:19.893152 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854efaac-74ab-4e0b-9584-e69f5931c51b None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:19.894910 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-854efaac-74ab-4e0b-9584-e69f5931c51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:19.994690 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854efaac-74ab-4e0b-9584-e69f5931c51b None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:20.006654 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3082de83-dd01-4710-b659-108fcf1f5788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:20.010034 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082de83-dd01-4710-b659-108fcf1f5788 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:20.010413 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082de83-dd01-4710-b659-108fcf1f5788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:20.010701 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3082de83-dd01-4710-b659-108fcf1f5788 admin admin] PATH_INFO: `/v3/users/72c5e6acbf1b46e6abec65e410ae89b5` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:20.012110 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3082de83-dd01-4710-b659-108fcf1f5788 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72c5e6acbf1b46e6abec65e410ae89b5)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:20.014551 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3082de83-dd01-4710-b659-108fcf1f5788 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:20.102609 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3082de83-dd01-4710-b659-108fcf1f5788 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72c5e6acbf1b46e6abec65e410ae89b5'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:20.104331 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3082de83-dd01-4710-b659-108fcf1f5788 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72c5e6acbf1b46e6abec65e410ae89b5'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:20.120518 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3082de83-dd01-4710-b659-108fcf1f5788 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72c5e6acbf1b46e6abec65e410ae89b5'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:20.121499 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3082de83-dd01-4710-b659-108fcf1f5788 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72c5e6acbf1b46e6abec65e410ae89b5'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:20.148708 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:20.161287 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-3082de83-dd01-4710-b659-108fcf1f5788 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72c5e6acbf1b46e6abec65e410ae89b5'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:20.190288 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1324/2625] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:31:19 2022] DELETE /identity/v3/users/72c5e6acbf1b46e6abec65e410ae89b5 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:20.195380 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7fa22-efaa-4ebb-80fe-a683dc2ca93c None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:20.196291 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7fa22-efaa-4ebb-80fe-a683dc2ca93c None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:20.196771 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce7fa22-efaa-4ebb-80fe-a683dc2ca93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:20.204699 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:20.231131 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-7ce7fa22-efaa-4ebb-80fe-a683dc2ca93c None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:20.232704 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-7ce7fa22-efaa-4ebb-80fe-a683dc2ca93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:20.336155 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1302/2626] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:20.352485 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3082de83-dd01-4710-b659-108fcf1f5788 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:20.354387 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-3082de83-dd01-4710-b659-108fcf1f5788 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:20.444483 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3082de83-dd01-4710-b659-108fcf1f5788 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:20.461034 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca4d872-154f-4ad5-bfce-91e167d8aeea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:20.462198 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca4d872-154f-4ad5-bfce-91e167d8aeea service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:20.462677 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca4d872-154f-4ad5-bfce-91e167d8aeea service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:20.463116 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca4d872-154f-4ad5-bfce-91e167d8aeea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:20.475251 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca4d872-154f-4ad5-bfce-91e167d8aeea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:20.477899 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca4d872-154f-4ad5-bfce-91e167d8aeea service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:20.493108 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1325/2627] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:20.863890 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce7fa22-efaa-4ebb-80fe-a683dc2ca93c None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:20.865553 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-7ce7fa22-efaa-4ebb-80fe-a683dc2ca93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:20.880869 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce7fa22-efaa-4ebb-80fe-a683dc2ca93c None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:20.886821 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d6c50f-e9f5-4562-afd2-7775e4034e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:20.887985 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6c50f-e9f5-4562-afd2-7775e4034e6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:20.888283 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6c50f-e9f5-4562-afd2-7775e4034e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:20.888710 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d6c50f-e9f5-4562-afd2-7775e4034e6b admin admin] PATH_INFO: `/v3/projects/ef7868209d6d4fb69a483fff24d2af19` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:20.890633 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d6c50f-e9f5-4562-afd2-7775e4034e6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef7868209d6d4fb69a483fff24d2af19)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:20.892932 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d6c50f-e9f5-4562-afd2-7775e4034e6b admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:20.962801 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-99d6c50f-e9f5-4562-afd2-7775e4034e6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef7868209d6d4fb69a483fff24d2af19', 'request_id': 'req-99d6c50f-e9f5-4562-afd2-7775e4034e6b'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:20.973937 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-99d6c50f-e9f5-4562-afd2-7775e4034e6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef7868209d6d4fb69a483fff24d2af19', 'request_id': 'req-99d6c50f-e9f5-4562-afd2-7775e4034e6b'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:20.986399 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-99d6c50f-e9f5-4562-afd2-7775e4034e6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef7868209d6d4fb69a483fff24d2af19', 'request_id': 'req-99d6c50f-e9f5-4562-afd2-7775e4034e6b'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:21.019037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-99d6c50f-e9f5-4562-afd2-7775e4034e6b admin admin] The token cache is being invalidate because project ef7868209d6d4fb69a483fff24d2af19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:31:21.019691 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-99d6c50f-e9f5-4562-afd2-7775e4034e6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:21.021749 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-99d6c50f-e9f5-4562-afd2-7775e4034e6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:21.023863 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1303/2628] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:31:20 2022] DELETE /identity/v3/projects/ef7868209d6d4fb69a483fff24d2af19 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:21.028551 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca4d872-154f-4ad5-bfce-91e167d8aeea service neutron] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:21.046582 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca4d872-154f-4ad5-bfce-91e167d8aeea service neutron] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:21.129677 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b554d6ab-5dda-4bfe-b2bd-da5a781a5630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:21.131141 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554d6ab-5dda-4bfe-b2bd-da5a781a5630 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:21.131673 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554d6ab-5dda-4bfe-b2bd-da5a781a5630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:21.132903 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554d6ab-5dda-4bfe-b2bd-da5a781a5630 admin admin] PATH_INFO: `/v3/users/856c52bdabef473abb6699509dcb6a7c` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:21.135069 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b554d6ab-5dda-4bfe-b2bd-da5a781a5630 admin admin] RBAC: Authorizing `identity:delete_user(user_id=856c52bdabef473abb6699509dcb6a7c)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:21.138448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b554d6ab-5dda-4bfe-b2bd-da5a781a5630 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:21.212998 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-b554d6ab-5dda-4bfe-b2bd-da5a781a5630 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '856c52bdabef473abb6699509dcb6a7c'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:21.212998 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-b554d6ab-5dda-4bfe-b2bd-da5a781a5630 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '856c52bdabef473abb6699509dcb6a7c'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:21.221717 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-b554d6ab-5dda-4bfe-b2bd-da5a781a5630 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '856c52bdabef473abb6699509dcb6a7c'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:21.222867 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-b554d6ab-5dda-4bfe-b2bd-da5a781a5630 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '856c52bdabef473abb6699509dcb6a7c'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:21.238854 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-b554d6ab-5dda-4bfe-b2bd-da5a781a5630 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '856c52bdabef473abb6699509dcb6a7c'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:21.256218 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1326/2629] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:31:21 2022] DELETE /identity/v3/users/856c52bdabef473abb6699509dcb6a7c => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:31:21.598906 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d6c50f-e9f5-4562-afd2-7775e4034e6b admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:21.600602 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-99d6c50f-e9f5-4562-afd2-7775e4034e6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:21.627769 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d6c50f-e9f5-4562-afd2-7775e4034e6b admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:21.809578 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8ffed-3ee6-45d3-a291-e5c50ab19692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:21.810490 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8ffed-3ee6-45d3-a291-e5c50ab19692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:21.810809 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8ffed-3ee6-45d3-a291-e5c50ab19692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:21.811043 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8ffed-3ee6-45d3-a291-e5c50ab19692 admin admin] PATH_INFO: `/v3/projects/f68ca1558b724506b3542f8cfc917e6e` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:21.813117 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d8ffed-3ee6-45d3-a291-e5c50ab19692 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f68ca1558b724506b3542f8cfc917e6e)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:21.818726 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d8ffed-3ee6-45d3-a291-e5c50ab19692 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:21.921084 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:21.996498 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b554d6ab-5dda-4bfe-b2bd-da5a781a5630 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:21.998862 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-b554d6ab-5dda-4bfe-b2bd-da5a781a5630 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:22.010161 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-c0d8ffed-3ee6-45d3-a291-e5c50ab19692 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f68ca1558b724506b3542f8cfc917e6e', 'request_id': 'req-c0d8ffed-3ee6-45d3-a291-e5c50ab19692'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:22.011524 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:22.022692 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b554d6ab-5dda-4bfe-b2bd-da5a781a5630 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:22.023066 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-c0d8ffed-3ee6-45d3-a291-e5c50ab19692 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f68ca1558b724506b3542f8cfc917e6e', 'request_id': 'req-c0d8ffed-3ee6-45d3-a291-e5c50ab19692'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:22.038581 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-c0d8ffed-3ee6-45d3-a291-e5c50ab19692 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f68ca1558b724506b3542f8cfc917e6e', 'request_id': 'req-c0d8ffed-3ee6-45d3-a291-e5c50ab19692'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:22.065410 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-c0d8ffed-3ee6-45d3-a291-e5c50ab19692 admin admin] The token cache is being invalidate because project f68ca1558b724506b3542f8cfc917e6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:31:22.065863 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-c0d8ffed-3ee6-45d3-a291-e5c50ab19692 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:22.067380 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-c0d8ffed-3ee6-45d3-a291-e5c50ab19692 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:22.070587 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1304/2630] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:31:21 2022] DELETE /identity/v3/projects/f68ca1558b724506b3542f8cfc917e6e => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:22.076850 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5c9c2-b900-47df-ba86-50c9c02aab20 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:22.077155 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5c9c2-b900-47df-ba86-50c9c02aab20 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:22.077432 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5c9c2-b900-47df-ba86-50c9c02aab20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:22.121606 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-1dd5c9c2-b900-47df-ba86-50c9c02aab20 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:22.123546 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-1dd5c9c2-b900-47df-ba86-50c9c02aab20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:22.175468 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8eb13-76cb-4d16-898d-914c9de83b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:22.176982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8eb13-76cb-4d16-898d-914c9de83b14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:22.177556 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8eb13-76cb-4d16-898d-914c9de83b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:22.178020 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8eb13-76cb-4d16-898d-914c9de83b14 admin admin] PATH_INFO: `/v3/users/9f096c73af1f420e87cb3e4de1e8e8b9` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:22.180836 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a8eb13-76cb-4d16-898d-914c9de83b14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f096c73af1f420e87cb3e4de1e8e8b9)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:22.187269 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a8eb13-76cb-4d16-898d-914c9de83b14 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:22.285869 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1305/2631] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:22.298490 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146248ec-4254-4d33-8882-0bfb844d45b7 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:22.298490 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146248ec-4254-4d33-8882-0bfb844d45b7 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:22.298490 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146248ec-4254-4d33-8882-0bfb844d45b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:22.309535 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-c8a8eb13-76cb-4d16-898d-914c9de83b14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f096c73af1f420e87cb3e4de1e8e8b9'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:22.310209 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-c8a8eb13-76cb-4d16-898d-914c9de83b14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f096c73af1f420e87cb3e4de1e8e8b9'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:22.322013 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-146248ec-4254-4d33-8882-0bfb844d45b7 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:22.324265 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-146248ec-4254-4d33-8882-0bfb844d45b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:22.325027 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-c8a8eb13-76cb-4d16-898d-914c9de83b14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f096c73af1f420e87cb3e4de1e8e8b9'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:22.325817 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-c8a8eb13-76cb-4d16-898d-914c9de83b14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f096c73af1f420e87cb3e4de1e8e8b9'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:22.339957 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1306/2632] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:22.346405 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146248ec-4254-4d33-8882-0bfb844d45b7 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:22.349618 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-c8a8eb13-76cb-4d16-898d-914c9de83b14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f096c73af1f420e87cb3e4de1e8e8b9'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:22.365406 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146248ec-4254-4d33-8882-0bfb844d45b7 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:22.375626 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1327/2633] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:31:21 2022] DELETE /identity/v3/users/9f096c73af1f420e87cb3e4de1e8e8b9 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:22.376279 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93513e1-0baf-4e20-a134-8f2f4ebb97fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:22.376999 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93513e1-0baf-4e20-a134-8f2f4ebb97fe admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:22.377277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93513e1-0baf-4e20-a134-8f2f4ebb97fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:22.377547 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93513e1-0baf-4e20-a134-8f2f4ebb97fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:22.378971 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93513e1-0baf-4e20-a134-8f2f4ebb97fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:31:22.379797 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93513e1-0baf-4e20-a134-8f2f4ebb97fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:22.380938 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb145c8-cce6-458b-95b2-595ecf90987f None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:22.381124 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93513e1-0baf-4e20-a134-8f2f4ebb97fe admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:22.381597 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb145c8-cce6-458b-95b2-595ecf90987f None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:22.382402 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb145c8-cce6-458b-95b2-595ecf90987f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:22.401607 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1307/2634] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:31:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:31:22.402642 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93513e1-0baf-4e20-a134-8f2f4ebb97fe admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:22.413635 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-2eb145c8-cce6-458b-95b2-595ecf90987f None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:22.414267 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93513e1-0baf-4e20-a134-8f2f4ebb97fe admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:22.415924 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-2eb145c8-cce6-458b-95b2-595ecf90987f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:22.507451 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453c14c9-feb4-47e1-868e-fa695f3e54fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:22.508018 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c14c9-feb4-47e1-868e-fa695f3e54fe admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:22.508298 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c14c9-feb4-47e1-868e-fa695f3e54fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:22.508904 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c14c9-feb4-47e1-868e-fa695f3e54fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:22.509834 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453c14c9-feb4-47e1-868e-fa695f3e54fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:22.512244 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453c14c9-feb4-47e1-868e-fa695f3e54fe admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:22.543511 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1328/2635] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:31:22.545329 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb145c8-cce6-458b-95b2-595ecf90987f None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:22.547888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-2eb145c8-cce6-458b-95b2-595ecf90987f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:22.558204 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1308/2636] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:31:22 2022] POST /identity/v3/projects => generated 385 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:22.562718 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453c14c9-feb4-47e1-868e-fa695f3e54fe admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:22.595834 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:22.610576 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:22.671153 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb145c8-cce6-458b-95b2-595ecf90987f None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:22.680099 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453c14c9-feb4-47e1-868e-fa695f3e54fe admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:22.690161 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efdcd68-30df-4ec1-b5d2-ab8f366ad1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:22.691346 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efdcd68-30df-4ec1-b5d2-ab8f366ad1a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:22.691780 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efdcd68-30df-4ec1-b5d2-ab8f366ad1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:22.692212 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efdcd68-30df-4ec1-b5d2-ab8f366ad1a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:22.694790 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efdcd68-30df-4ec1-b5d2-ab8f366ad1a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:22.699393 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efdcd68-30df-4ec1-b5d2-ab8f366ad1a2 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:22.727941 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1309/2637] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:31:22 2022] POST /identity/v3/users => generated 293 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:22.729554 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efdcd68-30df-4ec1-b5d2-ab8f366ad1a2 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:22.741164 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efdcd68-30df-4ec1-b5d2-ab8f366ad1a2 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:22.746324 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13893c33-820b-4f76-8441-420d54faa332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:22.747017 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13893c33-820b-4f76-8441-420d54faa332 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:22.747294 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13893c33-820b-4f76-8441-420d54faa332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:22.747571 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13893c33-820b-4f76-8441-420d54faa332 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:22.748331 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13893c33-820b-4f76-8441-420d54faa332 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:22.750224 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13893c33-820b-4f76-8441-420d54faa332 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:22.766813 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1310/2638] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:22.767685 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13893c33-820b-4f76-8441-420d54faa332 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:22.779780 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13893c33-820b-4f76-8441-420d54faa332 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:22.786310 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e29390-383f-4fee-b1a1-bb9241686c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:22.787201 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e29390-383f-4fee-b1a1-bb9241686c63 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:22.787460 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e29390-383f-4fee-b1a1-bb9241686c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:22.787737 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e29390-383f-4fee-b1a1-bb9241686c63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:22.788565 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e29390-383f-4fee-b1a1-bb9241686c63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:22.788765 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b5a871-806a-48db-869b-25532a32cfe2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:22.789508 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b5a871-806a-48db-869b-25532a32cfe2 service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:22.789793 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b5a871-806a-48db-869b-25532a32cfe2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:22.790069 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b5a871-806a-48db-869b-25532a32cfe2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:22.790375 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e29390-383f-4fee-b1a1-bb9241686c63 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:22.807505 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1311/2639] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:22.808449 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e29390-383f-4fee-b1a1-bb9241686c63 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:22.809688 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b5a871-806a-48db-869b-25532a32cfe2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:22.811217 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b5a871-806a-48db-869b-25532a32cfe2 service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:22.819851 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e29390-383f-4fee-b1a1-bb9241686c63 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:22.832718 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b73088-5bf5-42a6-a9ab-5e07278d797f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:22.834111 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b73088-5bf5-42a6-a9ab-5e07278d797f admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:22.834111 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b73088-5bf5-42a6-a9ab-5e07278d797f admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:22.834385 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b73088-5bf5-42a6-a9ab-5e07278d797f admin admin] PATH_INFO: `/v3/projects/8fb8dfcdf4064f2ba862e21ca621c763/users/c1ac35d6153048d18946c38df9318210/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:22.836806 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1329/2640] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:22.857334 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b73088-5bf5-42a6-a9ab-5e07278d797f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fb8dfcdf4064f2ba862e21ca621c763, user_id=c1ac35d6153048d18946c38df9318210, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:22.860392 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b73088-5bf5-42a6-a9ab-5e07278d797f admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:22.882749 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1312/2641] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:31:22 2022] PUT /identity/v3/projects/8fb8dfcdf4064f2ba862e21ca621c763/users/c1ac35d6153048d18946c38df9318210/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:22.894755 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b3cb6-ddc5-41fa-aaae-4364c82dcd0a None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:22.895058 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b3cb6-ddc5-41fa-aaae-4364c82dcd0a None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:22.895336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b3cb6-ddc5-41fa-aaae-4364c82dcd0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:22.954567 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-021b3cb6-ddc5-41fa-aaae-4364c82dcd0a None None] MFA Rules not processed for user `c1ac35d6153048d18946c38df9318210`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:22.955815 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-021b3cb6-ddc5-41fa-aaae-4364c82dcd0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:23.059115 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1330/2642] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:23.060170 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b73088-5bf5-42a6-a9ab-5e07278d797f admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:23.230581 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b73088-5bf5-42a6-a9ab-5e07278d797f admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:23.241136 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d471c-b7d6-4c08-b441-edd997e518f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:23.242070 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d471c-b7d6-4c08-b441-edd997e518f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:23.243056 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d471c-b7d6-4c08-b441-edd997e518f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:23.243339 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2d471c-b7d6-4c08-b441-edd997e518f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:23.246369 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d471c-b7d6-4c08-b441-edd997e518f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:23.250468 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2d471c-b7d6-4c08-b441-edd997e518f7 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:23.315596 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1313/2643] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:31:23 2022] POST /identity/v3/projects => generated 383 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:23.322404 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b3cb6-ddc5-41fa-aaae-4364c82dcd0a None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:23.340815 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b3cb6-ddc5-41fa-aaae-4364c82dcd0a None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:23.421714 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594f543c-b128-4643-8191-de93eda9f3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:23.422963 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f543c-b128-4643-8191-de93eda9f3bd admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:23.423473 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f543c-b128-4643-8191-de93eda9f3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:23.424123 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f543c-b128-4643-8191-de93eda9f3bd admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:23.425288 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594f543c-b128-4643-8191-de93eda9f3bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:23.428178 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594f543c-b128-4643-8191-de93eda9f3bd admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:23.453175 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:23.461225 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1331/2644] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:31:23 2022] POST /identity/v3/users => generated 298 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:23.474185 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d471c-b7d6-4c08-b441-edd997e518f7 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:23.492206 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2d471c-b7d6-4c08-b441-edd997e518f7 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:23.498105 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df79a2d2-834d-4e62-af5b-dde8dec946e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:23.498969 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df79a2d2-834d-4e62-af5b-dde8dec946e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:23.499295 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df79a2d2-834d-4e62-af5b-dde8dec946e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:23.499573 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df79a2d2-834d-4e62-af5b-dde8dec946e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:23.500831 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df79a2d2-834d-4e62-af5b-dde8dec946e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:23.502583 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df79a2d2-834d-4e62-af5b-dde8dec946e0 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:23.522987 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1314/2645] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:23 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:23.523924 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594f543c-b128-4643-8191-de93eda9f3bd admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:23.543239 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594f543c-b128-4643-8191-de93eda9f3bd admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:23.564387 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4164b618-4ba7-4d39-b532-9f1f21de6722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:23.565271 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164b618-4ba7-4d39-b532-9f1f21de6722 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:23.566128 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164b618-4ba7-4d39-b532-9f1f21de6722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:23.566128 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164b618-4ba7-4d39-b532-9f1f21de6722 admin admin] PATH_INFO: `/v3/projects/3e762c6cc891491cbfefb02a462dbc88/users/19d25ea9ee1a4691b082dc99a1218d6c/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:23.598717 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4164b618-4ba7-4d39-b532-9f1f21de6722 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e762c6cc891491cbfefb02a462dbc88, user_id=19d25ea9ee1a4691b082dc99a1218d6c, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:23.603332 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4164b618-4ba7-4d39-b532-9f1f21de6722 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:23.616515 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df79a2d2-834d-4e62-af5b-dde8dec946e0 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:23.618310 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-df79a2d2-834d-4e62-af5b-dde8dec946e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:23.636207 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1332/2646] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:31:23 2022] PUT /identity/v3/projects/3e762c6cc891491cbfefb02a462dbc88/users/19d25ea9ee1a4691b082dc99a1218d6c/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:23.640800 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4164b618-4ba7-4d39-b532-9f1f21de6722 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:23.642010 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df79a2d2-834d-4e62-af5b-dde8dec946e0 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:23.733221 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:23.771299 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ec767b-054e-4734-934c-6a6bbfa1da62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:23.772840 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec767b-054e-4734-934c-6a6bbfa1da62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:23.773411 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec767b-054e-4734-934c-6a6bbfa1da62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:23.773982 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec767b-054e-4734-934c-6a6bbfa1da62 admin admin] PATH_INFO: `/v3/projects/b03f25d58f5b4ec58f91134036777cfe` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:23.776516 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ec767b-054e-4734-934c-6a6bbfa1da62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b03f25d58f5b4ec58f91134036777cfe)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:23.778929 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ec767b-054e-4734-934c-6a6bbfa1da62 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:23.814158 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4164b618-4ba7-4d39-b532-9f1f21de6722 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:23.820991 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4677044-7836-4565-ab8d-2a3ffec41c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:23.821940 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4677044-7836-4565-ab8d-2a3ffec41c5e admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:23.822469 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4677044-7836-4565-ab8d-2a3ffec41c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:23.823039 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4677044-7836-4565-ab8d-2a3ffec41c5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:23.823971 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4677044-7836-4565-ab8d-2a3ffec41c5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:23.825884 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4677044-7836-4565-ab8d-2a3ffec41c5e admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:23.852336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1333/2647] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:23 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:23.855081 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4677044-7836-4565-ab8d-2a3ffec41c5e admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:23.870443 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-a9ec767b-054e-4734-934c-6a6bbfa1da62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b03f25d58f5b4ec58f91134036777cfe', 'request_id': 'req-a9ec767b-054e-4734-934c-6a6bbfa1da62'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:23.874461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4677044-7836-4565-ab8d-2a3ffec41c5e admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:23.878254 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-a9ec767b-054e-4734-934c-6a6bbfa1da62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b03f25d58f5b4ec58f91134036777cfe', 'request_id': 'req-a9ec767b-054e-4734-934c-6a6bbfa1da62'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:23.888285 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-a9ec767b-054e-4734-934c-6a6bbfa1da62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b03f25d58f5b4ec58f91134036777cfe', 'request_id': 'req-a9ec767b-054e-4734-934c-6a6bbfa1da62'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:23.911249 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-a9ec767b-054e-4734-934c-6a6bbfa1da62 admin admin] The token cache is being invalidate because project b03f25d58f5b4ec58f91134036777cfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:31:23.911888 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-a9ec767b-054e-4734-934c-6a6bbfa1da62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:23.912751 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-a9ec767b-054e-4734-934c-6a6bbfa1da62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:23.918817 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1315/2648] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:31:23 2022] DELETE /identity/v3/projects/b03f25d58f5b4ec58f91134036777cfe => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:23.920140 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ec767b-054e-4734-934c-6a6bbfa1da62 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:23.923451 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-a9ec767b-054e-4734-934c-6a6bbfa1da62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:23.994569 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4be2a6-8476-4ba1-90ee-b23ced153b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:23.995941 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4be2a6-8476-4ba1-90ee-b23ced153b05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:23.996366 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4be2a6-8476-4ba1-90ee-b23ced153b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:23.996647 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4be2a6-8476-4ba1-90ee-b23ced153b05 admin admin] PATH_INFO: `/v3/projects/3e762c6cc891491cbfefb02a462dbc88/users/19d25ea9ee1a4691b082dc99a1218d6c/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:23.999602 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4be2a6-8476-4ba1-90ee-b23ced153b05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e762c6cc891491cbfefb02a462dbc88, user_id=19d25ea9ee1a4691b082dc99a1218d6c, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:24.005590 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4be2a6-8476-4ba1-90ee-b23ced153b05 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:24.032938 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1334/2649] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:31:23 2022] PUT /identity/v3/projects/3e762c6cc891491cbfefb02a462dbc88/users/19d25ea9ee1a4691b082dc99a1218d6c/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:24.036034 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce5f2cc-cbe7-4bfe-a63f-50f7d2e2b7eb None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:24.036326 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce5f2cc-cbe7-4bfe-a63f-50f7d2e2b7eb None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:24.036637 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce5f2cc-cbe7-4bfe-a63f-50f7d2e2b7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:24.087147 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ec767b-054e-4734-934c-6a6bbfa1da62 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:24.094421 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-fce5f2cc-cbe7-4bfe-a63f-50f7d2e2b7eb None None] MFA Rules not processed for user `19d25ea9ee1a4691b082dc99a1218d6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:24.095904 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-fce5f2cc-cbe7-4bfe-a63f-50f7d2e2b7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:24.216682 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:24.217311 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39164792-3f70-4d60-8b8c-c0b215bc30f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:24.218225 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39164792-3f70-4d60-8b8c-c0b215bc30f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:24.218497 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39164792-3f70-4d60-8b8c-c0b215bc30f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:24.218765 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39164792-3f70-4d60-8b8c-c0b215bc30f4 admin admin] PATH_INFO: `/v3/users/8605e7e981e8497d833b46c0394c19b3` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:24.221217 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39164792-3f70-4d60-8b8c-c0b215bc30f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8605e7e981e8497d833b46c0394c19b3)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:24.223806 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39164792-3f70-4d60-8b8c-c0b215bc30f4 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:24.276594 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1335/2650] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:24 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:31:24.280644 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:24.281588 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce5f2cc-cbe7-4bfe-a63f-50f7d2e2b7eb None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:24.284321 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-fce5f2cc-cbe7-4bfe-a63f-50f7d2e2b7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:24.347807 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-39164792-3f70-4d60-8b8c-c0b215bc30f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8605e7e981e8497d833b46c0394c19b3'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:24.373015 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-39164792-3f70-4d60-8b8c-c0b215bc30f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8605e7e981e8497d833b46c0394c19b3'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:24.386068 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-39164792-3f70-4d60-8b8c-c0b215bc30f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8605e7e981e8497d833b46c0394c19b3'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:24.391978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce5f2cc-cbe7-4bfe-a63f-50f7d2e2b7eb None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:24.402018 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd50d75-0f1d-4f03-a540-f15d73a5610e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:24.403000 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd50d75-0f1d-4f03-a540-f15d73a5610e service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:24.403516 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd50d75-0f1d-4f03-a540-f15d73a5610e service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:24.403796 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd50d75-0f1d-4f03-a540-f15d73a5610e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:24.406213 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-fbd50d75-0f1d-4f03-a540-f15d73a5610e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:24.406606 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-39164792-3f70-4d60-8b8c-c0b215bc30f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8605e7e981e8497d833b46c0394c19b3'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:24.408680 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-39164792-3f70-4d60-8b8c-c0b215bc30f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8605e7e981e8497d833b46c0394c19b3'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:24.421042 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1316/2651] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:31:23 2022] DELETE /identity/v3/users/8605e7e981e8497d833b46c0394c19b3 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:24.518709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd50d75-0f1d-4f03-a540-f15d73a5610e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:24.522068 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd50d75-0f1d-4f03-a540-f15d73a5610e service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:24.537010 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1336/2652] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:24.998988 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39164792-3f70-4d60-8b8c-c0b215bc30f4 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:25.000480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-39164792-3f70-4d60-8b8c-c0b215bc30f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:25.236843 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39164792-3f70-4d60-8b8c-c0b215bc30f4 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:25.252547 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7812c07d-2026-402b-8198-797c9fa164ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:25.254007 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812c07d-2026-402b-8198-797c9fa164ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:25.254574 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812c07d-2026-402b-8198-797c9fa164ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:25.254952 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812c07d-2026-402b-8198-797c9fa164ed admin admin] PATH_INFO: `/v3/projects/8afcf689df434505b5c327d610e7a53f` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:25.256700 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7812c07d-2026-402b-8198-797c9fa164ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=8afcf689df434505b5c327d610e7a53f)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:25.264338 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7812c07d-2026-402b-8198-797c9fa164ed admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:25.401511 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-7812c07d-2026-402b-8198-797c9fa164ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8afcf689df434505b5c327d610e7a53f', 'request_id': 'req-7812c07d-2026-402b-8198-797c9fa164ed'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:25.417523 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-7812c07d-2026-402b-8198-797c9fa164ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8afcf689df434505b5c327d610e7a53f', 'request_id': 'req-7812c07d-2026-402b-8198-797c9fa164ed'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:25.433336 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-7812c07d-2026-402b-8198-797c9fa164ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8afcf689df434505b5c327d610e7a53f', 'request_id': 'req-7812c07d-2026-402b-8198-797c9fa164ed'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:25.467906 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-7812c07d-2026-402b-8198-797c9fa164ed admin admin] The token cache is being invalidate because project 8afcf689df434505b5c327d610e7a53f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:31:25.468300 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-7812c07d-2026-402b-8198-797c9fa164ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:25.471662 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-7812c07d-2026-402b-8198-797c9fa164ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:25.473829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1317/2653] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:31:24 2022] DELETE /identity/v3/projects/8afcf689df434505b5c327d610e7a53f => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:25.480000 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339b266-d81e-43c8-9605-9783584321f7 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:25.480532 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339b266-d81e-43c8-9605-9783584321f7 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:25.480755 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339b266-d81e-43c8-9605-9783584321f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:25.499849 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-9339b266-d81e-43c8-9605-9783584321f7 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:25.501389 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-9339b266-d81e-43c8-9605-9783584321f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:25.576862 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1337/2654] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:25.584283 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12667f22-a1e4-4497-a238-fa6f94bb3cba None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:25.584570 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12667f22-a1e4-4497-a238-fa6f94bb3cba None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:25.584853 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12667f22-a1e4-4497-a238-fa6f94bb3cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:25.603823 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-12667f22-a1e4-4497-a238-fa6f94bb3cba None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:25.605155 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-12667f22-a1e4-4497-a238-fa6f94bb3cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:25.613232 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1318/2655] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:25.617654 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9339b266-d81e-43c8-9605-9783584321f7 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:25.628297 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9339b266-d81e-43c8-9605-9783584321f7 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:25.633509 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348e541-5ef6-4a97-a113-e94d064cf289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:25.634207 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348e541-5ef6-4a97-a113-e94d064cf289 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:25.634487 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348e541-5ef6-4a97-a113-e94d064cf289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:25.634889 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d348e541-5ef6-4a97-a113-e94d064cf289 admin admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:25.636335 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d348e541-5ef6-4a97-a113-e94d064cf289 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:31:25.636646 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d348e541-5ef6-4a97-a113-e94d064cf289 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:25.638308 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d348e541-5ef6-4a97-a113-e94d064cf289 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:25.656990 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1338/2656] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:31:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:31:25.658174 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12667f22-a1e4-4497-a238-fa6f94bb3cba None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:25.669505 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12667f22-a1e4-4497-a238-fa6f94bb3cba None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:25.674436 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe5c48b-b254-4e5e-aeda-5562f54bba75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:25.675121 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5c48b-b254-4e5e-aeda-5562f54bba75 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:25.675390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5c48b-b254-4e5e-aeda-5562f54bba75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:25.675660 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5c48b-b254-4e5e-aeda-5562f54bba75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:25.676339 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe5c48b-b254-4e5e-aeda-5562f54bba75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:25.678273 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe5c48b-b254-4e5e-aeda-5562f54bba75 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:25.700848 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:DELETE=15,keystone:SELECT=82,keystone:INSERT=3 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:25.720655 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348e541-5ef6-4a97-a113-e94d064cf289 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:25.721647 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1319/2657] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:31:25 2022] POST /identity/v3/projects => generated 365 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:25.736966 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d348e541-5ef6-4a97-a113-e94d064cf289 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:25.783506 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=100 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:25.823386 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48448128-c8d1-4979-970e-47a4faa8ee09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:25.824312 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48448128-c8d1-4979-970e-47a4faa8ee09 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:25.825510 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48448128-c8d1-4979-970e-47a4faa8ee09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:25.825869 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48448128-c8d1-4979-970e-47a4faa8ee09 admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:25.826693 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48448128-c8d1-4979-970e-47a4faa8ee09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:25.829349 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48448128-c8d1-4979-970e-47a4faa8ee09 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:25.858571 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1339/2658] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:31:25 2022] POST /identity/v3/users => generated 283 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:31:25.862970 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe5c48b-b254-4e5e-aeda-5562f54bba75 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:25.879098 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe5c48b-b254-4e5e-aeda-5562f54bba75 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:25.890991 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7375d92b-63ab-49ff-9757-f73b03bfa940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:25.892737 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375d92b-63ab-49ff-9757-f73b03bfa940 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:25.893146 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375d92b-63ab-49ff-9757-f73b03bfa940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:25.894004 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7375d92b-63ab-49ff-9757-f73b03bfa940 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:25.894845 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7375d92b-63ab-49ff-9757-f73b03bfa940 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:25.897756 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7375d92b-63ab-49ff-9757-f73b03bfa940 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:25.920001 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1320/2659] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:25.922254 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48448128-c8d1-4979-970e-47a4faa8ee09 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:25.940627 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48448128-c8d1-4979-970e-47a4faa8ee09 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:25.950712 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d4b11-8e3c-4ba4-af07-5953f3fd5a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:25.953316 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d4b11-8e3c-4ba4-af07-5953f3fd5a34 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:25.954221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d4b11-8e3c-4ba4-af07-5953f3fd5a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:25.955161 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d4b11-8e3c-4ba4-af07-5953f3fd5a34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:25.956629 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80d4b11-8e3c-4ba4-af07-5953f3fd5a34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:25.959527 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80d4b11-8e3c-4ba4-af07-5953f3fd5a34 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:25.973524 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1340/2660] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:25.977903 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7375d92b-63ab-49ff-9757-f73b03bfa940 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:26.002198 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7375d92b-63ab-49ff-9757-f73b03bfa940 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:26.016024 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb58671-511c-439d-b87e-53a57574ea0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:26.019371 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb58671-511c-439d-b87e-53a57574ea0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:26.019657 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb58671-511c-439d-b87e-53a57574ea0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:26.020021 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb58671-511c-439d-b87e-53a57574ea0a admin admin] PATH_INFO: `/v3/projects/5dd604209b694f82a5f6a3400d64344d/users/024e0fa788334c048f722be3477f3795/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:26.054382 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb58671-511c-439d-b87e-53a57574ea0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dd604209b694f82a5f6a3400d64344d, user_id=024e0fa788334c048f722be3477f3795, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:26.059835 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb58671-511c-439d-b87e-53a57574ea0a admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:26.093202 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1321/2661] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:31:25 2022] PUT /identity/v3/projects/5dd604209b694f82a5f6a3400d64344d/users/024e0fa788334c048f722be3477f3795/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:26.102875 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058a30a-8aac-4d80-8d64-8f3749fbc1e9 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:26.103452 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058a30a-8aac-4d80-8d64-8f3749fbc1e9 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:26.105336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058a30a-8aac-4d80-8d64-8f3749fbc1e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:26.158364 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-6058a30a-8aac-4d80-8d64-8f3749fbc1e9 None None] MFA Rules not processed for user `024e0fa788334c048f722be3477f3795`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:26.160025 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-6058a30a-8aac-4d80-8d64-8f3749fbc1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:26.309607 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1341/2662] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:26 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:26.321208 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb58671-511c-439d-b87e-53a57574ea0a admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:26.480024 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb58671-511c-439d-b87e-53a57574ea0a admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:26.485164 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c47ed6a-410a-41a1-a296-53bdf2adeb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:26.486005 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47ed6a-410a-41a1-a296-53bdf2adeb41 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:26.486730 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47ed6a-410a-41a1-a296-53bdf2adeb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:26.487164 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47ed6a-410a-41a1-a296-53bdf2adeb41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:26.488060 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c47ed6a-410a-41a1-a296-53bdf2adeb41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:26.490364 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c47ed6a-410a-41a1-a296-53bdf2adeb41 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:26.561647 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1322/2663] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:31:26 2022] POST /identity/v3/projects => generated 365 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:26.565869 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058a30a-8aac-4d80-8d64-8f3749fbc1e9 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:26.582892 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058a30a-8aac-4d80-8d64-8f3749fbc1e9 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:26.720210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d1f98-a0e4-4f71-9885-716ac603ceb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:26.720907 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d1f98-a0e4-4f71-9885-716ac603ceb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:26.721238 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d1f98-a0e4-4f71-9885-716ac603ceb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:26.721514 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d1f98-a0e4-4f71-9885-716ac603ceb9 admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:26.725870 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d1f98-a0e4-4f71-9885-716ac603ceb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:26.725870 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d1f98-a0e4-4f71-9885-716ac603ceb9 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:26.755341 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1342/2664] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:31:26 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:26.760689 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c47ed6a-410a-41a1-a296-53bdf2adeb41 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:26.777199 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c47ed6a-410a-41a1-a296-53bdf2adeb41 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:26.791402 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6bc4c8-36b5-44b7-aa18-9dc93016fcd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:26.792534 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6bc4c8-36b5-44b7-aa18-9dc93016fcd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:26.793084 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6bc4c8-36b5-44b7-aa18-9dc93016fcd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:26.793349 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6bc4c8-36b5-44b7-aa18-9dc93016fcd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:26.797870 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6bc4c8-36b5-44b7-aa18-9dc93016fcd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:26.802496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6bc4c8-36b5-44b7-aa18-9dc93016fcd7 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:26.821983 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1323/2665] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:31:26.827208 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d1f98-a0e4-4f71-9885-716ac603ceb9 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:26.839130 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d1f98-a0e4-4f71-9885-716ac603ceb9 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:26.850037 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd86ca-1009-477f-9023-5526de3361a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:26.851055 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd86ca-1009-477f-9023-5526de3361a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:26.851988 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd86ca-1009-477f-9023-5526de3361a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:26.852344 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd86ca-1009-477f-9023-5526de3361a5 admin admin] PATH_INFO: `/v3/projects/b6f969c195294d19a3f719299a570c71/users/016814a120c047c084ca57285da98129/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:26.875667 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd86ca-1009-477f-9023-5526de3361a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6f969c195294d19a3f719299a570c71, user_id=016814a120c047c084ca57285da98129, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:26.881219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd86ca-1009-477f-9023-5526de3361a5 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:26.906571 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1343/2666] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:31:26 2022] PUT /identity/v3/projects/b6f969c195294d19a3f719299a570c71/users/016814a120c047c084ca57285da98129/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:31:26.910403 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6bc4c8-36b5-44b7-aa18-9dc93016fcd7 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:26.927684 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6bc4c8-36b5-44b7-aa18-9dc93016fcd7 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:27.019204 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44bdce2-6cdb-4bf2-91c2-0f938ecb6f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:27.020013 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44bdce2-6cdb-4bf2-91c2-0f938ecb6f02 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:27.020291 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44bdce2-6cdb-4bf2-91c2-0f938ecb6f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:27.020585 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44bdce2-6cdb-4bf2-91c2-0f938ecb6f02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:27.021374 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44bdce2-6cdb-4bf2-91c2-0f938ecb6f02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:27.022983 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44bdce2-6cdb-4bf2-91c2-0f938ecb6f02 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:27.034552 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1324/2667] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:27.038955 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd86ca-1009-477f-9023-5526de3361a5 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:27.050180 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd86ca-1009-477f-9023-5526de3361a5 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:27.054689 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d1e4c-b5b7-43d6-9029-d083b62020b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:27.055579 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d1e4c-b5b7-43d6-9029-d083b62020b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:27.055853 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d1e4c-b5b7-43d6-9029-d083b62020b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:27.056129 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0d1e4c-b5b7-43d6-9029-d083b62020b5 admin admin] PATH_INFO: `/v3/projects/b6f969c195294d19a3f719299a570c71/users/016814a120c047c084ca57285da98129/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:27.058218 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0d1e4c-b5b7-43d6-9029-d083b62020b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6f969c195294d19a3f719299a570c71, user_id=016814a120c047c084ca57285da98129, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:27.060867 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0d1e4c-b5b7-43d6-9029-d083b62020b5 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:27.063233 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:27.074286 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1344/2668] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:31:27 2022] PUT /identity/v3/projects/b6f969c195294d19a3f719299a570c71/users/016814a120c047c084ca57285da98129/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:27.085829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca17954-590d-4aa0-ae11-ee8ebc356ca2 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:27.086224 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca17954-590d-4aa0-ae11-ee8ebc356ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:27.086567 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca17954-590d-4aa0-ae11-ee8ebc356ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:27.115075 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-dca17954-590d-4aa0-ae11-ee8ebc356ca2 None None] MFA Rules not processed for user `016814a120c047c084ca57285da98129`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:27.116332 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-dca17954-590d-4aa0-ae11-ee8ebc356ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:27.139512 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:27.205313 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1325/2669] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:27 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:27.217382 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d1e4c-b5b7-43d6-9029-d083b62020b5 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:27.219517 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-cc0d1e4c-b5b7-43d6-9029-d083b62020b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:27.294471 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0d1e4c-b5b7-43d6-9029-d083b62020b5 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:27.300674 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4911c9-16b0-4eba-9ef5-1927431a3c50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:27.300801 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4911c9-16b0-4eba-9ef5-1927431a3c50 service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:27.301047 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4911c9-16b0-4eba-9ef5-1927431a3c50 service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:27.301324 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4911c9-16b0-4eba-9ef5-1927431a3c50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:27.315960 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4911c9-16b0-4eba-9ef5-1927431a3c50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:27.317950 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4911c9-16b0-4eba-9ef5-1927431a3c50 service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:27.330166 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1345/2670] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:27 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:27.419371 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca17954-590d-4aa0-ae11-ee8ebc356ca2 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:27.429479 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca17954-590d-4aa0-ae11-ee8ebc356ca2 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:27.433272 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1cf1de-7df1-4bda-adb3-b4367482b03c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:27.433925 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1cf1de-7df1-4bda-adb3-b4367482b03c service cinder] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:27.434208 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1cf1de-7df1-4bda-adb3-b4367482b03c service cinder] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:27.434487 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1cf1de-7df1-4bda-adb3-b4367482b03c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:27.500686 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1cf1de-7df1-4bda-adb3-b4367482b03c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:27.502822 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1cf1de-7df1-4bda-adb3-b4367482b03c service cinder] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:27.515404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1326/2671] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:27 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:29.524212 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4911c9-16b0-4eba-9ef5-1927431a3c50 service cinder] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:29.525732 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-8b4911c9-16b0-4eba-9ef5-1927431a3c50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:29.537171 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4911c9-16b0-4eba-9ef5-1927431a3c50 service cinder] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:29.541207 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de61d4-f7e3-4406-96ee-1465bace50ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:29.542342 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de61d4-f7e3-4406-96ee-1465bace50ef service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:29.542472 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de61d4-f7e3-4406-96ee-1465bace50ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:29.542881 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de61d4-f7e3-4406-96ee-1465bace50ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:29.544743 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-32de61d4-f7e3-4406-96ee-1465bace50ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:29.610035 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de61d4-f7e3-4406-96ee-1465bace50ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:29.611475 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de61d4-f7e3-4406-96ee-1465bace50ef service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:29.621727 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1346/2672] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:29 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:33.888559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1cf1de-7df1-4bda-adb3-b4367482b03c service cinder] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:33.890086 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-9d1cf1de-7df1-4bda-adb3-b4367482b03c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:33.969249 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1cf1de-7df1-4bda-adb3-b4367482b03c service cinder] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:33.974619 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c790156b-b5e7-4e96-8e4f-90b2cda0dc81 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30963f2f464e4f54a8aea0175370a235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:33.975330 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790156b-b5e7-4e96-8e4f-90b2cda0dc81 service placement] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:33.975612 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790156b-b5e7-4e96-8e4f-90b2cda0dc81 service placement] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:33.975897 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c790156b-b5e7-4e96-8e4f-90b2cda0dc81 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:33.977889 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-c790156b-b5e7-4e96-8e4f-90b2cda0dc81 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:33.990614 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c790156b-b5e7-4e96-8e4f-90b2cda0dc81 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:33.992286 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c790156b-b5e7-4e96-8e4f-90b2cda0dc81 service placement] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:34.016178 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1327/2673] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:35.243239 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38faf5-4134-45b5-bb77-6d3e7232be40 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:35.243796 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38faf5-4134-45b5-bb77-6d3e7232be40 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:35.243796 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38faf5-4134-45b5-bb77-6d3e7232be40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:35.262919 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-1b38faf5-4134-45b5-bb77-6d3e7232be40 None None] MFA Rules not processed for user `86a18e313e4944748aef58d54bfe7b13`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:35.264029 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-1b38faf5-4134-45b5-bb77-6d3e7232be40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:35.331926 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1347/2674] 10.222.0.50 () {58 vars in 967 bytes} [Fri May 27 03:31:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:35.346627 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c790156b-b5e7-4e96-8e4f-90b2cda0dc81 service placement] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:35.348450 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-c790156b-b5e7-4e96-8e4f-90b2cda0dc81 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:35.407364 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:35.429118 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c790156b-b5e7-4e96-8e4f-90b2cda0dc81 service placement] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:35.436380 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcf89b8-74b6-466f-8238-e3264a774528 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de53e2f0fba4e0fac70887067c1963d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:35.437073 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf89b8-74b6-466f-8238-e3264a774528 service swift] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:35.437612 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf89b8-74b6-466f-8238-e3264a774528 service swift] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:35.437892 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf89b8-74b6-466f-8238-e3264a774528 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:35.450436 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcf89b8-74b6-466f-8238-e3264a774528 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:35.452055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcf89b8-74b6-466f-8238-e3264a774528 service swift] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:35.462679 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1328/2675] 10.222.0.50 () {60 vars in 1384 bytes} [Fri May 27 03:31:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 03:31:36.164861 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38faf5-4134-45b5-bb77-6d3e7232be40 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:36.226881 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:36.283082 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38faf5-4134-45b5-bb77-6d3e7232be40 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:36.289681 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ca2227-f7d8-4aa0-8c27-4888057d21a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:36.290940 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ca2227-f7d8-4aa0-8c27-4888057d21a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:36.291602 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ca2227-f7d8-4aa0-8c27-4888057d21a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:36.291986 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ca2227-f7d8-4aa0-8c27-4888057d21a5 admin admin] PATH_INFO: `/v3/users/024e0fa788334c048f722be3477f3795` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:36.300612 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ca2227-f7d8-4aa0-8c27-4888057d21a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=024e0fa788334c048f722be3477f3795)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:36.307135 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ca2227-f7d8-4aa0-8c27-4888057d21a5 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:36.399327 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-73ca2227-f7d8-4aa0-8c27-4888057d21a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '024e0fa788334c048f722be3477f3795'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:36.400352 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-73ca2227-f7d8-4aa0-8c27-4888057d21a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '024e0fa788334c048f722be3477f3795'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:36.413446 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-73ca2227-f7d8-4aa0-8c27-4888057d21a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '024e0fa788334c048f722be3477f3795'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:36.414857 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-73ca2227-f7d8-4aa0-8c27-4888057d21a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '024e0fa788334c048f722be3477f3795'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:36.432786 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-73ca2227-f7d8-4aa0-8c27-4888057d21a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '024e0fa788334c048f722be3477f3795'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:36.447970 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1348/2676] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:31:36 2022] DELETE /identity/v3/users/024e0fa788334c048f722be3477f3795 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:31:36.453139 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7fb8f2-15c3-4175-9287-38c070a32989 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:36.453677 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7fb8f2-15c3-4175-9287-38c070a32989 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:36.453925 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7fb8f2-15c3-4175-9287-38c070a32989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:36.474042 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-9c7fb8f2-15c3-4175-9287-38c070a32989 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:36.475533 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-9c7fb8f2-15c3-4175-9287-38c070a32989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:36.569800 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1329/2677] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:36.582847 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ca2227-f7d8-4aa0-8c27-4888057d21a5 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:36.584656 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-73ca2227-f7d8-4aa0-8c27-4888057d21a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:36.675904 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ca2227-f7d8-4aa0-8c27-4888057d21a5 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:36.680562 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efed5dc-9c46-4b87-afd4-fe56fab71acb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:36.681239 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efed5dc-9c46-4b87-afd4-fe56fab71acb service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:36.681511 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efed5dc-9c46-4b87-afd4-fe56fab71acb service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:36.681783 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efed5dc-9c46-4b87-afd4-fe56fab71acb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:36.694285 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efed5dc-9c46-4b87-afd4-fe56fab71acb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:36.695741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efed5dc-9c46-4b87-afd4-fe56fab71acb service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:36.723985 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1349/2678] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:37.348522 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7fb8f2-15c3-4175-9287-38c070a32989 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:37.350415 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-9c7fb8f2-15c3-4175-9287-38c070a32989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:37.364344 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7fb8f2-15c3-4175-9287-38c070a32989 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:37.383310 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69819c3-4880-4818-b809-8d807602a341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:37.386875 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69819c3-4880-4818-b809-8d807602a341 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:37.387390 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69819c3-4880-4818-b809-8d807602a341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:37.388103 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69819c3-4880-4818-b809-8d807602a341 admin admin] PATH_INFO: `/v3/projects/5dd604209b694f82a5f6a3400d64344d` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:37.389669 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69819c3-4880-4818-b809-8d807602a341 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dd604209b694f82a5f6a3400d64344d)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:37.391801 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69819c3-4880-4818-b809-8d807602a341 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:37.469502 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-d69819c3-4880-4818-b809-8d807602a341 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dd604209b694f82a5f6a3400d64344d', 'request_id': 'req-d69819c3-4880-4818-b809-8d807602a341'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:37.478453 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-d69819c3-4880-4818-b809-8d807602a341 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dd604209b694f82a5f6a3400d64344d', 'request_id': 'req-d69819c3-4880-4818-b809-8d807602a341'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:37.492878 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-d69819c3-4880-4818-b809-8d807602a341 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dd604209b694f82a5f6a3400d64344d', 'request_id': 'req-d69819c3-4880-4818-b809-8d807602a341'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:37.520780 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-d69819c3-4880-4818-b809-8d807602a341 admin admin] The token cache is being invalidate because project 5dd604209b694f82a5f6a3400d64344d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:31:37.521222 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-d69819c3-4880-4818-b809-8d807602a341 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:37.522173 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-d69819c3-4880-4818-b809-8d807602a341 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:37.524625 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1330/2679] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:31:37 2022] DELETE /identity/v3/projects/5dd604209b694f82a5f6a3400d64344d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:37.528648 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efed5dc-9c46-4b87-afd4-fe56fab71acb service neutron] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:37.539552 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efed5dc-9c46-4b87-afd4-fe56fab71acb service neutron] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:37.642592 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:37.657306 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a2158-5bb3-45d6-8036-32336da074e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:37.659944 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a2158-5bb3-45d6-8036-32336da074e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:37.660388 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a2158-5bb3-45d6-8036-32336da074e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:37.661252 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a2158-5bb3-45d6-8036-32336da074e1 admin admin] PATH_INFO: `/v3/users/016814a120c047c084ca57285da98129` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:37.663619 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a2158-5bb3-45d6-8036-32336da074e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=016814a120c047c084ca57285da98129)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:37.666079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a2158-5bb3-45d6-8036-32336da074e1 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:37.747100 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f68a2158-5bb3-45d6-8036-32336da074e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '016814a120c047c084ca57285da98129'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:37.747989 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f68a2158-5bb3-45d6-8036-32336da074e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '016814a120c047c084ca57285da98129'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:37.764262 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f68a2158-5bb3-45d6-8036-32336da074e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '016814a120c047c084ca57285da98129'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:37.765198 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f68a2158-5bb3-45d6-8036-32336da074e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '016814a120c047c084ca57285da98129'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:37.787580 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f68a2158-5bb3-45d6-8036-32336da074e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '016814a120c047c084ca57285da98129'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:37.827348 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1350/2680] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:31:37 2022] DELETE /identity/v3/users/016814a120c047c084ca57285da98129 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:38.322823 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69819c3-4880-4818-b809-8d807602a341 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:38.328777 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-d69819c3-4880-4818-b809-8d807602a341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:38.354356 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69819c3-4880-4818-b809-8d807602a341 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:38.505326 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3f258-673a-4cef-ae59-8ba518dee58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:38.507002 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3f258-673a-4cef-ae59-8ba518dee58d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:38.507602 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3f258-673a-4cef-ae59-8ba518dee58d admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:38.508429 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3f258-673a-4cef-ae59-8ba518dee58d admin admin] PATH_INFO: `/v3/projects/b6f969c195294d19a3f719299a570c71` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:38.514885 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa3f258-673a-4cef-ae59-8ba518dee58d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6f969c195294d19a3f719299a570c71)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:38.520808 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa3f258-673a-4cef-ae59-8ba518dee58d admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:38.524184 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:38.677606 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-ffa3f258-673a-4cef-ae59-8ba518dee58d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6f969c195294d19a3f719299a570c71', 'request_id': 'req-ffa3f258-673a-4cef-ae59-8ba518dee58d'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:38.699703 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-ffa3f258-673a-4cef-ae59-8ba518dee58d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6f969c195294d19a3f719299a570c71', 'request_id': 'req-ffa3f258-673a-4cef-ae59-8ba518dee58d'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:38.724044 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-ffa3f258-673a-4cef-ae59-8ba518dee58d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6f969c195294d19a3f719299a570c71', 'request_id': 'req-ffa3f258-673a-4cef-ae59-8ba518dee58d'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:38.758047 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-ffa3f258-673a-4cef-ae59-8ba518dee58d admin admin] The token cache is being invalidate because project b6f969c195294d19a3f719299a570c71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:31:38.758657 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-ffa3f258-673a-4cef-ae59-8ba518dee58d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:38.761353 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-ffa3f258-673a-4cef-ae59-8ba518dee58d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:38.764486 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1331/2681] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:31:38 2022] DELETE /identity/v3/projects/b6f969c195294d19a3f719299a570c71 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:38.770091 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba0cca8-ddcc-4eb4-ba20-eefde9ba7387 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:38.770613 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba0cca8-ddcc-4eb4-ba20-eefde9ba7387 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:38.770826 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba0cca8-ddcc-4eb4-ba20-eefde9ba7387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:38.808408 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-cba0cca8-ddcc-4eb4-ba20-eefde9ba7387 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:38.820089 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-cba0cca8-ddcc-4eb4-ba20-eefde9ba7387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:38.976633 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6effa-fd33-4621-a23b-e32319208d35 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:38.979474 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6effa-fd33-4621-a23b-e32319208d35 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:38.979915 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6effa-fd33-4621-a23b-e32319208d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:38.996888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1351/2682] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:39.021503 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-2ec6effa-fd33-4621-a23b-e32319208d35 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:39.023239 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-2ec6effa-fd33-4621-a23b-e32319208d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:39.040517 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba0cca8-ddcc-4eb4-ba20-eefde9ba7387 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:39.047356 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1332/2683] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:39.051762 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba0cca8-ddcc-4eb4-ba20-eefde9ba7387 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:39.058241 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab1fea-9b6f-490a-8c58-a6723b3f9acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:39.059570 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab1fea-9b6f-490a-8c58-a6723b3f9acd admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:39.060015 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab1fea-9b6f-490a-8c58-a6723b3f9acd admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:39.060620 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab1fea-9b6f-490a-8c58-a6723b3f9acd admin admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:39.061575 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dab1fea-9b6f-490a-8c58-a6723b3f9acd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:31:39.062059 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dab1fea-9b6f-490a-8c58-a6723b3f9acd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:39.063998 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dab1fea-9b6f-490a-8c58-a6723b3f9acd admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:39.100848 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1352/2684] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:31:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:31:39.105201 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6effa-fd33-4621-a23b-e32319208d35 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:39.119740 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6effa-fd33-4621-a23b-e32319208d35 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:39.127498 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86117b0-1f49-498f-9d64-3e1e27eb8c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:39.128166 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86117b0-1f49-498f-9d64-3e1e27eb8c5a admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:39.128442 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86117b0-1f49-498f-9d64-3e1e27eb8c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:39.128721 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86117b0-1f49-498f-9d64-3e1e27eb8c5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:39.129501 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86117b0-1f49-498f-9d64-3e1e27eb8c5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:39.131640 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86117b0-1f49-498f-9d64-3e1e27eb8c5a admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:39.191160 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab1fea-9b6f-490a-8c58-a6723b3f9acd admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:39.193964 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-1dab1fea-9b6f-490a-8c58-a6723b3f9acd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:39.200458 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1333/2685] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:31:39 2022] POST /identity/v3/projects => generated 379 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:39.201525 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86117b0-1f49-498f-9d64-3e1e27eb8c5a admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:39.219059 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab1fea-9b6f-490a-8c58-a6723b3f9acd admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:39.310463 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86117b0-1f49-498f-9d64-3e1e27eb8c5a admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:39.315579 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92536f8c-59e4-4831-9ca2-35656f84b712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:39.317303 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92536f8c-59e4-4831-9ca2-35656f84b712 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:39.317698 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92536f8c-59e4-4831-9ca2-35656f84b712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:39.318050 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92536f8c-59e4-4831-9ca2-35656f84b712 admin admin] PATH_INFO: `/v3/users/c1ac35d6153048d18946c38df9318210` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:39.321618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16dc7d1-d214-45a5-a0c2-4e15e2fb0a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:39.322337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16dc7d1-d214-45a5-a0c2-4e15e2fb0a80 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:39.322618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16dc7d1-d214-45a5-a0c2-4e15e2fb0a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:39.322893 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16dc7d1-d214-45a5-a0c2-4e15e2fb0a80 admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:39.324324 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16dc7d1-d214-45a5-a0c2-4e15e2fb0a80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:39.326578 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16dc7d1-d214-45a5-a0c2-4e15e2fb0a80 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:39.328440 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92536f8c-59e4-4831-9ca2-35656f84b712 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1ac35d6153048d18946c38df9318210)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:39.332574 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92536f8c-59e4-4831-9ca2-35656f84b712 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:39.343288 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:39.362689 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1334/2686] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:31:39 2022] POST /identity/v3/users => generated 290 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:39.363896 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16dc7d1-d214-45a5-a0c2-4e15e2fb0a80 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:39.399864 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16dc7d1-d214-45a5-a0c2-4e15e2fb0a80 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:39.408078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284910e-4f39-4c77-96c8-d2837c0331ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:39.412440 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284910e-4f39-4c77-96c8-d2837c0331ac admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:39.413743 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284910e-4f39-4c77-96c8-d2837c0331ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:39.415569 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7284910e-4f39-4c77-96c8-d2837c0331ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:39.420510 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7284910e-4f39-4c77-96c8-d2837c0331ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:39.423538 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-92536f8c-59e4-4831-9ca2-35656f84b712 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1ac35d6153048d18946c38df9318210'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:39.424364 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-92536f8c-59e4-4831-9ca2-35656f84b712 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1ac35d6153048d18946c38df9318210'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:39.426362 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7284910e-4f39-4c77-96c8-d2837c0331ac admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:39.435385 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-92536f8c-59e4-4831-9ca2-35656f84b712 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1ac35d6153048d18946c38df9318210'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:39.437704 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-92536f8c-59e4-4831-9ca2-35656f84b712 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1ac35d6153048d18946c38df9318210'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:39.454906 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-92536f8c-59e4-4831-9ca2-35656f84b712 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1ac35d6153048d18946c38df9318210'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:39.467440 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1335/2687] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:39.472438 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1353/2688] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:31:39 2022] DELETE /identity/v3/users/c1ac35d6153048d18946c38df9318210 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:39.472619 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284910e-4f39-4c77-96c8-d2837c0331ac admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:39.475362 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36591ffa-567f-48f9-9cd8-1b5d3b65cb54 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:39.475756 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36591ffa-567f-48f9-9cd8-1b5d3b65cb54 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:39.476126 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36591ffa-567f-48f9-9cd8-1b5d3b65cb54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:39.490664 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-7284910e-4f39-4c77-96c8-d2837c0331ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:39.497180 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-36591ffa-567f-48f9-9cd8-1b5d3b65cb54 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:39.498547 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-36591ffa-567f-48f9-9cd8-1b5d3b65cb54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:39.523509 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7284910e-4f39-4c77-96c8-d2837c0331ac admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:39.583557 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1354/2689] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:39.596191 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36591ffa-567f-48f9-9cd8-1b5d3b65cb54 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:39.597915 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-36591ffa-567f-48f9-9cd8-1b5d3b65cb54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:39.601521 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:39.639636 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a14640-b20c-40d7-a219-9ef821a9d3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:39.640505 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a14640-b20c-40d7-a219-9ef821a9d3c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:39.640969 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a14640-b20c-40d7-a219-9ef821a9d3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:39.641348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a14640-b20c-40d7-a219-9ef821a9d3c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:39.642202 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a14640-b20c-40d7-a219-9ef821a9d3c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:39.643939 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a14640-b20c-40d7-a219-9ef821a9d3c9 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:39.661784 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1336/2690] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:39.665870 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a14640-b20c-40d7-a219-9ef821a9d3c9 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:39.674560 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36591ffa-567f-48f9-9cd8-1b5d3b65cb54 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:39.679272 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d2be2-99fa-444d-8653-78e5ef705854 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:39.679947 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d2be2-99fa-444d-8653-78e5ef705854 service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:39.680227 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d2be2-99fa-444d-8653-78e5ef705854 service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:39.680497 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d2be2-99fa-444d-8653-78e5ef705854 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:39.680900 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a14640-b20c-40d7-a219-9ef821a9d3c9 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:39.695115 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1d2be2-99fa-444d-8653-78e5ef705854 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:39.695115 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1d2be2-99fa-444d-8653-78e5ef705854 service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:39.706302 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1355/2691] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:31:39.718943 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b0dff6-e8ef-4dbf-b79f-2a5b7035dbe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:39.719970 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0dff6-e8ef-4dbf-b79f-2a5b7035dbe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:39.721424 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0dff6-e8ef-4dbf-b79f-2a5b7035dbe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:39.721785 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0dff6-e8ef-4dbf-b79f-2a5b7035dbe8 admin admin] PATH_INFO: `/v3/projects/485b4f0e22a0494cb2b10d2f82c888c8/users/45170d27f8e24ed6a48b0d9240ee3b20/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:39.754359 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b0dff6-e8ef-4dbf-b79f-2a5b7035dbe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=485b4f0e22a0494cb2b10d2f82c888c8, user_id=45170d27f8e24ed6a48b0d9240ee3b20, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:39.757359 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b0dff6-e8ef-4dbf-b79f-2a5b7035dbe8 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:39.796096 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1337/2692] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:31:39 2022] PUT /identity/v3/projects/485b4f0e22a0494cb2b10d2f82c888c8/users/45170d27f8e24ed6a48b0d9240ee3b20/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:39.797226 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c668e4-f059-4176-b30e-e88f81a4ae43 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:39.797759 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c668e4-f059-4176-b30e-e88f81a4ae43 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:39.797759 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c668e4-f059-4176-b30e-e88f81a4ae43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:39.860291 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-b3c668e4-f059-4176-b30e-e88f81a4ae43 None None] MFA Rules not processed for user `45170d27f8e24ed6a48b0d9240ee3b20`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:39.862060 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-b3c668e4-f059-4176-b30e-e88f81a4ae43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:39.891921 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:39.967329 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1356/2693] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:39.971800 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b0dff6-e8ef-4dbf-b79f-2a5b7035dbe8 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:40.141577 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b0dff6-e8ef-4dbf-b79f-2a5b7035dbe8 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:40.160853 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2a1e3-01f6-4b86-8ce2-f728fe663292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:40.162426 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2a1e3-01f6-4b86-8ce2-f728fe663292 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:40.162726 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2a1e3-01f6-4b86-8ce2-f728fe663292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:40.162994 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2a1e3-01f6-4b86-8ce2-f728fe663292 admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:40.163795 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf2a1e3-01f6-4b86-8ce2-f728fe663292 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:40.175286 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf2a1e3-01f6-4b86-8ce2-f728fe663292 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:40.242502 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1338/2694] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:31:39 2022] POST /identity/v3/projects => generated 379 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:40.246884 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c668e4-f059-4176-b30e-e88f81a4ae43 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:40.266149 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c668e4-f059-4176-b30e-e88f81a4ae43 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:40.325017 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2a1e3-01f6-4b86-8ce2-f728fe663292 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:40.326835 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-7bf2a1e3-01f6-4b86-8ce2-f728fe663292 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:40.461853 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2a1e3-01f6-4b86-8ce2-f728fe663292 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:40.463869 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a5f0ff-6e86-48fe-8849-1f17d831c7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:40.464564 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5f0ff-6e86-48fe-8849-1f17d831c7bd admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:40.464904 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5f0ff-6e86-48fe-8849-1f17d831c7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:40.465276 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5f0ff-6e86-48fe-8849-1f17d831c7bd admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:40.466074 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a5f0ff-6e86-48fe-8849-1f17d831c7bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:40.467998 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a5f0ff-6e86-48fe-8849-1f17d831c7bd admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:40.472410 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaebf68-e4c7-48ef-818a-deb5ea378353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:40.473706 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaebf68-e4c7-48ef-818a-deb5ea378353 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:40.474227 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaebf68-e4c7-48ef-818a-deb5ea378353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:40.474713 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaebf68-e4c7-48ef-818a-deb5ea378353 admin admin] PATH_INFO: `/v3/projects/8fb8dfcdf4064f2ba862e21ca621c763` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:40.476300 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaebf68-e4c7-48ef-818a-deb5ea378353 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fb8dfcdf4064f2ba862e21ca621c763)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:40.478692 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaebf68-e4c7-48ef-818a-deb5ea378353 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:40.524159 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1357/2695] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:31:40 2022] POST /identity/v3/users => generated 296 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:40.528697 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a5f0ff-6e86-48fe-8849-1f17d831c7bd admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:40.540818 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a5f0ff-6e86-48fe-8849-1f17d831c7bd admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:40.553468 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d569f-4c8c-4f72-8662-d7f470d3952b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:40.553918 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d569f-4c8c-4f72-8662-d7f470d3952b admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:40.554299 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d569f-4c8c-4f72-8662-d7f470d3952b admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:40.554809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d569f-4c8c-4f72-8662-d7f470d3952b admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:40.556858 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d569f-4c8c-4f72-8662-d7f470d3952b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:40.561169 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d569f-4c8c-4f72-8662-d7f470d3952b admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:40.565269 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:40.592007 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1358/2696] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:40 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:40.596398 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d569f-4c8c-4f72-8662-d7f470d3952b admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:40.599143 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-faaebf68-e4c7-48ef-818a-deb5ea378353 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fb8dfcdf4064f2ba862e21ca621c763', 'request_id': 'req-faaebf68-e4c7-48ef-818a-deb5ea378353'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:40.607934 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-faaebf68-e4c7-48ef-818a-deb5ea378353 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fb8dfcdf4064f2ba862e21ca621c763', 'request_id': 'req-faaebf68-e4c7-48ef-818a-deb5ea378353'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:40.623126 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d569f-4c8c-4f72-8662-d7f470d3952b admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:40.664229 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-faaebf68-e4c7-48ef-818a-deb5ea378353 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fb8dfcdf4064f2ba862e21ca621c763', 'request_id': 'req-faaebf68-e4c7-48ef-818a-deb5ea378353'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:40.701237 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-faaebf68-e4c7-48ef-818a-deb5ea378353 admin admin] The token cache is being invalidate because project 8fb8dfcdf4064f2ba862e21ca621c763 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:31:40.701536 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-faaebf68-e4c7-48ef-818a-deb5ea378353 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:40.707461 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-faaebf68-e4c7-48ef-818a-deb5ea378353 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:40.717052 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1339/2697] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:31:40 2022] DELETE /identity/v3/projects/8fb8dfcdf4064f2ba862e21ca621c763 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:31:40.725224 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaebf68-e4c7-48ef-818a-deb5ea378353 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:40.734011 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-faaebf68-e4c7-48ef-818a-deb5ea378353 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:40.755847 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b683cb6-7d89-49e1-9636-c5ecc96ebb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:40.759805 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b683cb6-7d89-49e1-9636-c5ecc96ebb35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:40.760347 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b683cb6-7d89-49e1-9636-c5ecc96ebb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:40.761061 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b683cb6-7d89-49e1-9636-c5ecc96ebb35 admin admin] PATH_INFO: `/v3/projects/0e2d10c92ebc4a8bb4be4960ee581ef0/users/43fbc2275e09406fb91e42180672fae5/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:40.803515 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b683cb6-7d89-49e1-9636-c5ecc96ebb35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e2d10c92ebc4a8bb4be4960ee581ef0, user_id=43fbc2275e09406fb91e42180672fae5, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:40.807818 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b683cb6-7d89-49e1-9636-c5ecc96ebb35 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:40.843230 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1359/2698] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:31:40 2022] PUT /identity/v3/projects/0e2d10c92ebc4a8bb4be4960ee581ef0/users/43fbc2275e09406fb91e42180672fae5/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:40.847699 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b683cb6-7d89-49e1-9636-c5ecc96ebb35 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:40.850409 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-6b683cb6-7d89-49e1-9636-c5ecc96ebb35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:40.892459 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:40.902265 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaebf68-e4c7-48ef-818a-deb5ea378353 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:40.998185 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b683cb6-7d89-49e1-9636-c5ecc96ebb35 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:41.016008 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd5448-0eed-407f-984f-129bf643ea3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:41.017084 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd5448-0eed-407f-984f-129bf643ea3e admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:41.017465 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd5448-0eed-407f-984f-129bf643ea3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:41.017863 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fd5448-0eed-407f-984f-129bf643ea3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:41.018720 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fd5448-0eed-407f-984f-129bf643ea3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:41.022461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fd5448-0eed-407f-984f-129bf643ea3e admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:41.047026 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1360/2699] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:40 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:41.071221 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd5448-0eed-407f-984f-129bf643ea3e admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:41.089401 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6553cc-7d59-405a-ba4c-c28ec7f6d916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:41.090348 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6553cc-7d59-405a-ba4c-c28ec7f6d916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:41.090657 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6553cc-7d59-405a-ba4c-c28ec7f6d916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:41.090941 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6553cc-7d59-405a-ba4c-c28ec7f6d916 admin admin] PATH_INFO: `/v3/users/19d25ea9ee1a4691b082dc99a1218d6c` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:41.094874 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6553cc-7d59-405a-ba4c-c28ec7f6d916 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19d25ea9ee1a4691b082dc99a1218d6c)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:41.096400 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6553cc-7d59-405a-ba4c-c28ec7f6d916 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:41.098867 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fd5448-0eed-407f-984f-129bf643ea3e admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:41.133575 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46832736-7fd9-48c6-a683-3939843e850f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:41.135751 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46832736-7fd9-48c6-a683-3939843e850f admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:41.138582 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46832736-7fd9-48c6-a683-3939843e850f admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:41.141237 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46832736-7fd9-48c6-a683-3939843e850f admin admin] PATH_INFO: `/v3/projects/0e2d10c92ebc4a8bb4be4960ee581ef0/users/43fbc2275e09406fb91e42180672fae5/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:41.175196 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46832736-7fd9-48c6-a683-3939843e850f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e2d10c92ebc4a8bb4be4960ee581ef0, user_id=43fbc2275e09406fb91e42180672fae5, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:41.190722 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46832736-7fd9-48c6-a683-3939843e850f admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:41.224748 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-ea6553cc-7d59-405a-ba4c-c28ec7f6d916 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19d25ea9ee1a4691b082dc99a1218d6c'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:41.282297 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-ea6553cc-7d59-405a-ba4c-c28ec7f6d916 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19d25ea9ee1a4691b082dc99a1218d6c'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:41.285451 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1361/2700] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:31:41 2022] PUT /identity/v3/projects/0e2d10c92ebc4a8bb4be4960ee581ef0/users/43fbc2275e09406fb91e42180672fae5/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:41.303711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82eaaa-26bf-42be-82e0-222a4fd7fc57 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:41.304002 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82eaaa-26bf-42be-82e0-222a4fd7fc57 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:41.304336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82eaaa-26bf-42be-82e0-222a4fd7fc57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:41.311092 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-ea6553cc-7d59-405a-ba4c-c28ec7f6d916 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19d25ea9ee1a4691b082dc99a1218d6c'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:41.320944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:41.325174 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-ea6553cc-7d59-405a-ba4c-c28ec7f6d916 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19d25ea9ee1a4691b082dc99a1218d6c'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:41.327775 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-ea6553cc-7d59-405a-ba4c-c28ec7f6d916 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19d25ea9ee1a4691b082dc99a1218d6c'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:41.339533 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1340/2701] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:31:40 2022] DELETE /identity/v3/users/19d25ea9ee1a4691b082dc99a1218d6c => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:41.342100 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-2d82eaaa-26bf-42be-82e0-222a4fd7fc57 None None] MFA Rules not processed for user `43fbc2275e09406fb91e42180672fae5`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:41.352730 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-2d82eaaa-26bf-42be-82e0-222a4fd7fc57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:41.490972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1362/2702] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:41 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:41.508263 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6553cc-7d59-405a-ba4c-c28ec7f6d916 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:41.509908 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-ea6553cc-7d59-405a-ba4c-c28ec7f6d916 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:41.609111 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6553cc-7d59-405a-ba4c-c28ec7f6d916 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:41.623208 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e769b-0f1f-4d09-94b5-e334e898fc21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:41.624095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e769b-0f1f-4d09-94b5-e334e898fc21 service cinder] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:41.624478 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e769b-0f1f-4d09-94b5-e334e898fc21 service cinder] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:41.624854 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e769b-0f1f-4d09-94b5-e334e898fc21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:41.652003 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e769b-0f1f-4d09-94b5-e334e898fc21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:41.654509 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373e769b-0f1f-4d09-94b5-e334e898fc21 service cinder] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:41.673429 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1341/2703] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:41 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:31:41.783210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d82eaaa-26bf-42be-82e0-222a4fd7fc57 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:41.785078 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-2d82eaaa-26bf-42be-82e0-222a4fd7fc57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:41.910956 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d82eaaa-26bf-42be-82e0-222a4fd7fc57 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:41.915959 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa42cc2-e084-48b6-8ab1-3651f190e4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:41.921643 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa42cc2-e084-48b6-8ab1-3651f190e4f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:41.921643 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa42cc2-e084-48b6-8ab1-3651f190e4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:41.921643 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa42cc2-e084-48b6-8ab1-3651f190e4f6 admin admin] PATH_INFO: `/v3/projects/3e762c6cc891491cbfefb02a462dbc88` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:41.921643 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa42cc2-e084-48b6-8ab1-3651f190e4f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e762c6cc891491cbfefb02a462dbc88)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:41.923175 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa42cc2-e084-48b6-8ab1-3651f190e4f6 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:41.982386 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:42.012091 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-dfa42cc2-e084-48b6-8ab1-3651f190e4f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e762c6cc891491cbfefb02a462dbc88', 'request_id': 'req-dfa42cc2-e084-48b6-8ab1-3651f190e4f6'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:42.022747 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-dfa42cc2-e084-48b6-8ab1-3651f190e4f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e762c6cc891491cbfefb02a462dbc88', 'request_id': 'req-dfa42cc2-e084-48b6-8ab1-3651f190e4f6'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:42.037074 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-dfa42cc2-e084-48b6-8ab1-3651f190e4f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e762c6cc891491cbfefb02a462dbc88', 'request_id': 'req-dfa42cc2-e084-48b6-8ab1-3651f190e4f6'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:42.052075 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-dfa42cc2-e084-48b6-8ab1-3651f190e4f6 admin admin] The token cache is being invalidate because project 3e762c6cc891491cbfefb02a462dbc88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:31:42.052364 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-dfa42cc2-e084-48b6-8ab1-3651f190e4f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:42.053064 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-dfa42cc2-e084-48b6-8ab1-3651f190e4f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:42.057300 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1363/2704] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:31:41 2022] DELETE /identity/v3/projects/3e762c6cc891491cbfefb02a462dbc88 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:42.062417 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c6114-708f-4449-be26-7a6eef6f4b70 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:42.062634 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c6114-708f-4449-be26-7a6eef6f4b70 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:42.062788 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4c6114-708f-4449-be26-7a6eef6f4b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:42.085717 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-df4c6114-708f-4449-be26-7a6eef6f4b70 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:42.087027 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-df4c6114-708f-4449-be26-7a6eef6f4b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:42.140134 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa42cc2-e084-48b6-8ab1-3651f190e4f6 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:42.141915 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-dfa42cc2-e084-48b6-8ab1-3651f190e4f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:42.174913 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1342/2705] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:42.184084 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733156d-0e18-4ffd-b016-9b6fa6d4d7b1 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:42.184638 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733156d-0e18-4ffd-b016-9b6fa6d4d7b1 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:42.185137 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733156d-0e18-4ffd-b016-9b6fa6d4d7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:42.210939 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-f733156d-0e18-4ffd-b016-9b6fa6d4d7b1 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:42.213619 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-f733156d-0e18-4ffd-b016-9b6fa6d4d7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:42.234417 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1343/2706] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:31:42.238500 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f733156d-0e18-4ffd-b016-9b6fa6d4d7b1 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:42.254329 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f733156d-0e18-4ffd-b016-9b6fa6d4d7b1 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:42.260394 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48108845-3e8a-4a44-a5ef-7453f7fdc983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:42.261260 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48108845-3e8a-4a44-a5ef-7453f7fdc983 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:42.261819 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48108845-3e8a-4a44-a5ef-7453f7fdc983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:42.262150 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48108845-3e8a-4a44-a5ef-7453f7fdc983 admin admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:42.262997 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48108845-3e8a-4a44-a5ef-7453f7fdc983 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:31:42.263404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48108845-3e8a-4a44-a5ef-7453f7fdc983 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:42.265159 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48108845-3e8a-4a44-a5ef-7453f7fdc983 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:42.277920 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa42cc2-e084-48b6-8ab1-3651f190e4f6 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:42.289383 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1344/2707] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:31:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:31:42.293831 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48108845-3e8a-4a44-a5ef-7453f7fdc983 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:42.294508 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e26b65-7da6-4b9a-a50c-6878a36ed835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:42.295588 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e26b65-7da6-4b9a-a50c-6878a36ed835 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:42.295872 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e26b65-7da6-4b9a-a50c-6878a36ed835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:42.296145 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e26b65-7da6-4b9a-a50c-6878a36ed835 admin admin] PATH_INFO: `/v3/users/45170d27f8e24ed6a48b0d9240ee3b20` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:42.299615 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e26b65-7da6-4b9a-a50c-6878a36ed835 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45170d27f8e24ed6a48b0d9240ee3b20)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:42.301781 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e26b65-7da6-4b9a-a50c-6878a36ed835 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:42.318626 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48108845-3e8a-4a44-a5ef-7453f7fdc983 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:42.326075 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf8917-bdee-42f8-a8ef-49fe93934c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:42.326921 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf8917-bdee-42f8-a8ef-49fe93934c89 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:42.327330 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf8917-bdee-42f8-a8ef-49fe93934c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:42.327709 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf8917-bdee-42f8-a8ef-49fe93934c89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:42.328524 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecf8917-bdee-42f8-a8ef-49fe93934c89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:42.332436 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecf8917-bdee-42f8-a8ef-49fe93934c89 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:42.407268 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1345/2708] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:31:42 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:42.408368 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf8917-bdee-42f8-a8ef-49fe93934c89 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:42.421552 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f4e26b65-7da6-4b9a-a50c-6878a36ed835 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45170d27f8e24ed6a48b0d9240ee3b20'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:42.422790 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f4e26b65-7da6-4b9a-a50c-6878a36ed835 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45170d27f8e24ed6a48b0d9240ee3b20'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:42.443658 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f4e26b65-7da6-4b9a-a50c-6878a36ed835 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45170d27f8e24ed6a48b0d9240ee3b20'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:42.443658 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f4e26b65-7da6-4b9a-a50c-6878a36ed835 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45170d27f8e24ed6a48b0d9240ee3b20'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:42.478523 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f4e26b65-7da6-4b9a-a50c-6878a36ed835 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45170d27f8e24ed6a48b0d9240ee3b20'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:42.494900 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:42.503300 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf8917-bdee-42f8-a8ef-49fe93934c89 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:42.508352 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1364/2709] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:31:42 2022] DELETE /identity/v3/users/45170d27f8e24ed6a48b0d9240ee3b20 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:42.514976 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96058b0f-8467-4679-838c-86f3338b29f5 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:42.515452 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96058b0f-8467-4679-838c-86f3338b29f5 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:42.515729 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96058b0f-8467-4679-838c-86f3338b29f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:42.516669 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b11b186-9ee7-47db-bd5d-ffae57b069e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:42.518624 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b11b186-9ee7-47db-bd5d-ffae57b069e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:42.519520 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b11b186-9ee7-47db-bd5d-ffae57b069e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:42.519935 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b11b186-9ee7-47db-bd5d-ffae57b069e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:42.521184 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b11b186-9ee7-47db-bd5d-ffae57b069e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:42.523754 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b11b186-9ee7-47db-bd5d-ffae57b069e1 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:42.546412 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-96058b0f-8467-4679-838c-86f3338b29f5 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:42.547687 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-96058b0f-8467-4679-838c-86f3338b29f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:42.565884 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1346/2710] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:31:42 2022] POST /identity/v3/users => generated 293 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:42.566913 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b11b186-9ee7-47db-bd5d-ffae57b069e1 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:42.568832 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-5b11b186-9ee7-47db-bd5d-ffae57b069e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:42.582917 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b11b186-9ee7-47db-bd5d-ffae57b069e1 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:42.595703 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:42.645577 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1365/2711] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:42.662534 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42e5c47-752f-4310-9822-8d042ba77198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:42.663291 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e5c47-752f-4310-9822-8d042ba77198 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:42.663632 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e5c47-752f-4310-9822-8d042ba77198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:42.663973 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e5c47-752f-4310-9822-8d042ba77198 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:42.664389 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96058b0f-8467-4679-838c-86f3338b29f5 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:42.664831 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42e5c47-752f-4310-9822-8d042ba77198 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:42.668218 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-96058b0f-8467-4679-838c-86f3338b29f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:42.681140 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42e5c47-752f-4310-9822-8d042ba77198 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:42.701308 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1347/2712] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:42.707464 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42e5c47-752f-4310-9822-8d042ba77198 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:42.720831 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42e5c47-752f-4310-9822-8d042ba77198 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:42.725756 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965290a6-2581-4e61-af52-530a07278440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:42.726445 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965290a6-2581-4e61-af52-530a07278440 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:42.726729 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965290a6-2581-4e61-af52-530a07278440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:42.727004 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965290a6-2581-4e61-af52-530a07278440 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:42.732476 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965290a6-2581-4e61-af52-530a07278440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:42.735808 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965290a6-2581-4e61-af52-530a07278440 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:42.751628 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1348/2713] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:42.755389 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965290a6-2581-4e61-af52-530a07278440 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:42.767484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96058b0f-8467-4679-838c-86f3338b29f5 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:42.773772 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d89aff-35b7-4174-b161-5c26a036846d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:42.774078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965290a6-2581-4e61-af52-530a07278440 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:42.777067 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d89aff-35b7-4174-b161-5c26a036846d service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:42.777564 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d89aff-35b7-4174-b161-5c26a036846d service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:42.778362 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d89aff-35b7-4174-b161-5c26a036846d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:42.780405 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b75e4b-f2bd-4c0f-a40f-2e8f4c380685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:42.781605 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b75e4b-f2bd-4c0f-a40f-2e8f4c380685 admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:42.781955 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b75e4b-f2bd-4c0f-a40f-2e8f4c380685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:42.782462 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b75e4b-f2bd-4c0f-a40f-2e8f4c380685 admin admin] PATH_INFO: `/v3/projects/cfb0f434e1b84a31890d33378c2fc8ee/users/ecc14b3653d54d3390144d876feed9ad/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:42.791101 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d89aff-35b7-4174-b161-5c26a036846d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:42.793203 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d89aff-35b7-4174-b161-5c26a036846d service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:42.800515 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b75e4b-f2bd-4c0f-a40f-2e8f4c380685 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfb0f434e1b84a31890d33378c2fc8ee, user_id=ecc14b3653d54d3390144d876feed9ad, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:42.803238 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b75e4b-f2bd-4c0f-a40f-2e8f4c380685 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:42.811426 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1366/2714] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:42.846393 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1349/2715] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:31:42 2022] PUT /identity/v3/projects/cfb0f434e1b84a31890d33378c2fc8ee/users/ecc14b3653d54d3390144d876feed9ad/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:42.852547 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a667f-64c1-4e35-9976-9000d6501fc8 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:42.852547 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a667f-64c1-4e35-9976-9000d6501fc8 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:42.852974 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a667f-64c1-4e35-9976-9000d6501fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:42.886575 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-c86a667f-64c1-4e35-9976-9000d6501fc8 None None] MFA Rules not processed for user `ecc14b3653d54d3390144d876feed9ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:42.887875 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-c86a667f-64c1-4e35-9976-9000d6501fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:43.029127 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1367/2716] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:43.031097 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b75e4b-f2bd-4c0f-a40f-2e8f4c380685 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:43.209748 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b75e4b-f2bd-4c0f-a40f-2e8f4c380685 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:43.222602 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c948e36a-4198-4cff-9853-1fd8a01f733d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:43.223331 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948e36a-4198-4cff-9853-1fd8a01f733d admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:43.223618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948e36a-4198-4cff-9853-1fd8a01f733d admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:43.223900 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948e36a-4198-4cff-9853-1fd8a01f733d admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:43.224612 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c948e36a-4198-4cff-9853-1fd8a01f733d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:43.226948 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c948e36a-4198-4cff-9853-1fd8a01f733d admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:43.306903 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1350/2717] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:31:43 2022] POST /identity/v3/projects => generated 383 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:31:43.311308 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a667f-64c1-4e35-9976-9000d6501fc8 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:43.334126 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a667f-64c1-4e35-9976-9000d6501fc8 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:43.425997 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c948e36a-4198-4cff-9853-1fd8a01f733d admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:43.429333 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-c948e36a-4198-4cff-9853-1fd8a01f733d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:43.455158 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e911f5-a228-48f9-93fe-c39e7692f6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:43.457109 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e911f5-a228-48f9-93fe-c39e7692f6ee admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:43.457394 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e911f5-a228-48f9-93fe-c39e7692f6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:43.457883 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e911f5-a228-48f9-93fe-c39e7692f6ee admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:43.458658 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e911f5-a228-48f9-93fe-c39e7692f6ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:43.460643 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e911f5-a228-48f9-93fe-c39e7692f6ee admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:43.485656 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:43.501957 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1368/2718] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:31:43 2022] POST /identity/v3/users => generated 292 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:43.505897 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e911f5-a228-48f9-93fe-c39e7692f6ee admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:43.515033 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:43.536406 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e911f5-a228-48f9-93fe-c39e7692f6ee admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:43.550190 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702812ff-30db-4eed-b50f-fac12c797260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:43.551015 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702812ff-30db-4eed-b50f-fac12c797260 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:43.551366 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702812ff-30db-4eed-b50f-fac12c797260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:43.552233 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702812ff-30db-4eed-b50f-fac12c797260 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:43.553493 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702812ff-30db-4eed-b50f-fac12c797260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:43.557507 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702812ff-30db-4eed-b50f-fac12c797260 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:43.569764 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c948e36a-4198-4cff-9853-1fd8a01f733d admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:43.574620 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5cbc36-b494-4efd-b679-913c18239704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:43.575913 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5cbc36-b494-4efd-b679-913c18239704 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:43.576391 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5cbc36-b494-4efd-b679-913c18239704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:43.576681 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5cbc36-b494-4efd-b679-913c18239704 admin admin] PATH_INFO: `/v3/projects/485b4f0e22a0494cb2b10d2f82c888c8` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:43.577898 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5cbc36-b494-4efd-b679-913c18239704 admin admin] RBAC: Authorizing `identity:delete_project(project_id=485b4f0e22a0494cb2b10d2f82c888c8)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:43.582691 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5cbc36-b494-4efd-b679-913c18239704 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:43.602330 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1369/2719] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:43.609246 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702812ff-30db-4eed-b50f-fac12c797260 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:43.628212 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702812ff-30db-4eed-b50f-fac12c797260 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:43.641491 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f25d1-0fbd-4d60-a5ef-5d1146ddaf66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:43.644752 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f25d1-0fbd-4d60-a5ef-5d1146ddaf66 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:43.645875 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f25d1-0fbd-4d60-a5ef-5d1146ddaf66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:43.646387 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f25d1-0fbd-4d60-a5ef-5d1146ddaf66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:43.648332 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f25d1-0fbd-4d60-a5ef-5d1146ddaf66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:43.651101 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0f25d1-0fbd-4d60-a5ef-5d1146ddaf66 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:43.676706 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1370/2720] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:43.677834 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f25d1-0fbd-4d60-a5ef-5d1146ddaf66 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:43.686244 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-aa5cbc36-b494-4efd-b679-913c18239704 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '485b4f0e22a0494cb2b10d2f82c888c8', 'request_id': 'req-aa5cbc36-b494-4efd-b679-913c18239704'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:43.697322 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0f25d1-0fbd-4d60-a5ef-5d1146ddaf66 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:43.716133 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-aa5cbc36-b494-4efd-b679-913c18239704 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '485b4f0e22a0494cb2b10d2f82c888c8', 'request_id': 'req-aa5cbc36-b494-4efd-b679-913c18239704'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:43.724331 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-aa5cbc36-b494-4efd-b679-913c18239704 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '485b4f0e22a0494cb2b10d2f82c888c8', 'request_id': 'req-aa5cbc36-b494-4efd-b679-913c18239704'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:43.760820 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-aa5cbc36-b494-4efd-b679-913c18239704 admin admin] The token cache is being invalidate because project 485b4f0e22a0494cb2b10d2f82c888c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:31:43.761510 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-aa5cbc36-b494-4efd-b679-913c18239704 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:43.763653 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-aa5cbc36-b494-4efd-b679-913c18239704 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:43.772278 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1351/2721] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:31:43 2022] DELETE /identity/v3/projects/485b4f0e22a0494cb2b10d2f82c888c8 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:43.777420 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5cbc36-b494-4efd-b679-913c18239704 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:43.779915 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-aa5cbc36-b494-4efd-b679-913c18239704 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:43.821709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199e87b8-2e84-4353-bec9-c1665d8bf537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:43.823068 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e87b8-2e84-4353-bec9-c1665d8bf537 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:43.823564 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e87b8-2e84-4353-bec9-c1665d8bf537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:43.824045 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199e87b8-2e84-4353-bec9-c1665d8bf537 admin admin] PATH_INFO: `/v3/projects/0585485ff5cc4758a44de84bc3974afd/users/ac666eb623f34c989593ccb0745000f2/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:43.860610 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199e87b8-2e84-4353-bec9-c1665d8bf537 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0585485ff5cc4758a44de84bc3974afd, user_id=ac666eb623f34c989593ccb0745000f2, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:43.865499 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199e87b8-2e84-4353-bec9-c1665d8bf537 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:43.889893 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1371/2722] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:31:43 2022] PUT /identity/v3/projects/0585485ff5cc4758a44de84bc3974afd/users/ac666eb623f34c989593ccb0745000f2/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:43.900189 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86881a1e-ba25-4552-9c1d-4a6c20c65a2c None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:43.900513 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86881a1e-ba25-4552-9c1d-4a6c20c65a2c None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:43.900721 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86881a1e-ba25-4552-9c1d-4a6c20c65a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:43.905349 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5cbc36-b494-4efd-b679-913c18239704 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:43.929613 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-86881a1e-ba25-4552-9c1d-4a6c20c65a2c None None] MFA Rules not processed for user `ac666eb623f34c989593ccb0745000f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:43.931338 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-86881a1e-ba25-4552-9c1d-4a6c20c65a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:43.995261 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48c64f3-95c6-4ea1-8137-58e0964d48b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:43.996286 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c64f3-95c6-4ea1-8137-58e0964d48b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:43.996572 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c64f3-95c6-4ea1-8137-58e0964d48b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:43.996990 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c64f3-95c6-4ea1-8137-58e0964d48b1 admin admin] PATH_INFO: `/v3/users/43fbc2275e09406fb91e42180672fae5` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:43.998479 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48c64f3-95c6-4ea1-8137-58e0964d48b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43fbc2275e09406fb91e42180672fae5)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:44.000519 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48c64f3-95c6-4ea1-8137-58e0964d48b1 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:44.017441 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:44.020096 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1372/2723] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:31:44.024866 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86881a1e-ba25-4552-9c1d-4a6c20c65a2c None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:44.026555 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-86881a1e-ba25-4552-9c1d-4a6c20c65a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:44.052186 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86881a1e-ba25-4552-9c1d-4a6c20c65a2c None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:44.059021 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db2eba0-0551-4468-b1f5-3ff4b84fff65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:44.059762 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2eba0-0551-4468-b1f5-3ff4b84fff65 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:44.059977 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2eba0-0551-4468-b1f5-3ff4b84fff65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:44.060376 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2eba0-0551-4468-b1f5-3ff4b84fff65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:44.061103 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db2eba0-0551-4468-b1f5-3ff4b84fff65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:44.064428 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db2eba0-0551-4468-b1f5-3ff4b84fff65 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:44.105981 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-d48c64f3-95c6-4ea1-8137-58e0964d48b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43fbc2275e09406fb91e42180672fae5'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:44.111311 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1373/2724] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:31:44 2022] POST /identity/v3/projects => generated 385 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:44.112794 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db2eba0-0551-4468-b1f5-3ff4b84fff65 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:44.137423 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-d48c64f3-95c6-4ea1-8137-58e0964d48b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43fbc2275e09406fb91e42180672fae5'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:44.158756 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:44.160160 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-d48c64f3-95c6-4ea1-8137-58e0964d48b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43fbc2275e09406fb91e42180672fae5'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:44.175755 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-d48c64f3-95c6-4ea1-8137-58e0964d48b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43fbc2275e09406fb91e42180672fae5'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:44.177334 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-d48c64f3-95c6-4ea1-8137-58e0964d48b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43fbc2275e09406fb91e42180672fae5'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:44.193210 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1352/2725] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:31:43 2022] DELETE /identity/v3/users/43fbc2275e09406fb91e42180672fae5 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:31:44.200984 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db2eba0-0551-4468-b1f5-3ff4b84fff65 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:44.315114 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b71dc-6a23-41c9-a559-c2e08cd88154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:44.315897 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b71dc-6a23-41c9-a559-c2e08cd88154 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:44.316139 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b71dc-6a23-41c9-a559-c2e08cd88154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:44.316408 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b71dc-6a23-41c9-a559-c2e08cd88154 admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:44.317427 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b71dc-6a23-41c9-a559-c2e08cd88154 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:44.320296 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b71dc-6a23-41c9-a559-c2e08cd88154 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:44.386729 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1374/2726] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:31:44 2022] POST /identity/v3/users => generated 299 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:44.388067 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48c64f3-95c6-4ea1-8137-58e0964d48b1 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:44.389998 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-d48c64f3-95c6-4ea1-8137-58e0964d48b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:44.452690 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48c64f3-95c6-4ea1-8137-58e0964d48b1 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:44.472117 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409afc4d-d8ed-4b26-ab8d-3e2f9c6acb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:44.472823 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409afc4d-d8ed-4b26-ab8d-3e2f9c6acb35 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:44.473134 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409afc4d-d8ed-4b26-ab8d-3e2f9c6acb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:44.473410 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409afc4d-d8ed-4b26-ab8d-3e2f9c6acb35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:44.474161 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409afc4d-d8ed-4b26-ab8d-3e2f9c6acb35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:44.475835 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409afc4d-d8ed-4b26-ab8d-3e2f9c6acb35 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:44.502803 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1353/2727] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:44.504105 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b71dc-6a23-41c9-a559-c2e08cd88154 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:44.519964 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b71dc-6a23-41c9-a559-c2e08cd88154 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:44.533696 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d37d3-b288-40d5-a723-a611e2e01daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:44.534630 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d37d3-b288-40d5-a723-a611e2e01daa admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:44.534912 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d37d3-b288-40d5-a723-a611e2e01daa admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:44.535191 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d37d3-b288-40d5-a723-a611e2e01daa admin admin] PATH_INFO: `/v3/projects/bc29662523c04e94b2c3e03b34b33f57/users/7fb3ab8dc0cb409a99e18cf5c61713be/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:44.578220 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d37d3-b288-40d5-a723-a611e2e01daa admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc29662523c04e94b2c3e03b34b33f57, user_id=7fb3ab8dc0cb409a99e18cf5c61713be, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:44.580584 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d37d3-b288-40d5-a723-a611e2e01daa admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:44.621674 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1375/2728] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:31:44 2022] PUT /identity/v3/projects/bc29662523c04e94b2c3e03b34b33f57/users/7fb3ab8dc0cb409a99e18cf5c61713be/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:31:44.625551 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409afc4d-d8ed-4b26-ab8d-3e2f9c6acb35 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:44.640089 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409afc4d-d8ed-4b26-ab8d-3e2f9c6acb35 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:44.763340 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4801650-0d6d-4112-b09b-d5178ae4fa2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:44.764941 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4801650-0d6d-4112-b09b-d5178ae4fa2f admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:44.765637 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4801650-0d6d-4112-b09b-d5178ae4fa2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:44.766186 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4801650-0d6d-4112-b09b-d5178ae4fa2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:44.767136 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4801650-0d6d-4112-b09b-d5178ae4fa2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:44.769525 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4801650-0d6d-4112-b09b-d5178ae4fa2f admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:44.786493 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1354/2729] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:44.791977 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d37d3-b288-40d5-a723-a611e2e01daa admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:44.815742 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d37d3-b288-40d5-a723-a611e2e01daa admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:44.821835 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ea73c-aff7-4738-befe-44185998882e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:44.822932 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ea73c-aff7-4738-befe-44185998882e admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:44.823332 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ea73c-aff7-4738-befe-44185998882e admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:44.823556 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ea73c-aff7-4738-befe-44185998882e admin admin] PATH_INFO: `/v3/projects/bc29662523c04e94b2c3e03b34b33f57/users/7fb3ab8dc0cb409a99e18cf5c61713be/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:44.827318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3ea73c-aff7-4738-befe-44185998882e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc29662523c04e94b2c3e03b34b33f57, user_id=7fb3ab8dc0cb409a99e18cf5c61713be, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:44.830011 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3ea73c-aff7-4738-befe-44185998882e admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:44.854642 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b65d25-721d-4d05-8b88-3ddea3c1b194 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:44.854985 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b65d25-721d-4d05-8b88-3ddea3c1b194 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:44.855298 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b65d25-721d-4d05-8b88-3ddea3c1b194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:44.858258 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1376/2730] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:31:44 2022] PUT /identity/v3/projects/bc29662523c04e94b2c3e03b34b33f57/users/7fb3ab8dc0cb409a99e18cf5c61713be/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:31:44.875456 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ea73c-aff7-4738-befe-44185998882e admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:44.877476 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-5c3ea73c-aff7-4738-befe-44185998882e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:44.939497 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-a7b65d25-721d-4d05-8b88-3ddea3c1b194 None None] MFA Rules not processed for user `7fb3ab8dc0cb409a99e18cf5c61713be`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:44.944651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-a7b65d25-721d-4d05-8b88-3ddea3c1b194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:45.099987 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ea73c-aff7-4738-befe-44185998882e admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:45.102266 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:45.125269 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d13f9-d28c-4872-834d-e0a8cbb00da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:45.128427 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d13f9-d28c-4872-834d-e0a8cbb00da8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:45.128841 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d13f9-d28c-4872-834d-e0a8cbb00da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:45.140702 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1355/2731] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:44 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:45.141394 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d13f9-d28c-4872-834d-e0a8cbb00da8 admin admin] PATH_INFO: `/v3/projects/0e2d10c92ebc4a8bb4be4960ee581ef0` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:45.143871 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4d13f9-d28c-4872-834d-e0a8cbb00da8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e2d10c92ebc4a8bb4be4960ee581ef0)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:45.146477 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4d13f9-d28c-4872-834d-e0a8cbb00da8 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:45.153985 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b65d25-721d-4d05-8b88-3ddea3c1b194 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:45.157452 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-a7b65d25-721d-4d05-8b88-3ddea3c1b194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:45.206522 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:45.250186 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-fa4d13f9-d28c-4872-834d-e0a8cbb00da8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e2d10c92ebc4a8bb4be4960ee581ef0', 'request_id': 'req-fa4d13f9-d28c-4872-834d-e0a8cbb00da8'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:45.263357 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-fa4d13f9-d28c-4872-834d-e0a8cbb00da8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e2d10c92ebc4a8bb4be4960ee581ef0', 'request_id': 'req-fa4d13f9-d28c-4872-834d-e0a8cbb00da8'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:45.286406 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-fa4d13f9-d28c-4872-834d-e0a8cbb00da8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e2d10c92ebc4a8bb4be4960ee581ef0', 'request_id': 'req-fa4d13f9-d28c-4872-834d-e0a8cbb00da8'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:45.290866 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b65d25-721d-4d05-8b88-3ddea3c1b194 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:45.307289 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-fa4d13f9-d28c-4872-834d-e0a8cbb00da8 admin admin] The token cache is being invalidate because project 0e2d10c92ebc4a8bb4be4960ee581ef0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:31:45.307496 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-fa4d13f9-d28c-4872-834d-e0a8cbb00da8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:45.308872 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-fa4d13f9-d28c-4872-834d-e0a8cbb00da8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:45.314759 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1377/2732] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:31:44 2022] DELETE /identity/v3/projects/0e2d10c92ebc4a8bb4be4960ee581ef0 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:45.317558 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc25963-568a-43af-805d-b7c01e9fce63 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:45.317871 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc25963-568a-43af-805d-b7c01e9fce63 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:45.318123 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc25963-568a-43af-805d-b7c01e9fce63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:45.350677 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-3bc25963-568a-43af-805d-b7c01e9fce63 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:45.351909 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-3bc25963-568a-43af-805d-b7c01e9fce63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:45.398887 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08230dce-97a1-4b34-9018-c46c2af6881d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:45.399555 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08230dce-97a1-4b34-9018-c46c2af6881d service cinder] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:45.399838 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08230dce-97a1-4b34-9018-c46c2af6881d service cinder] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:45.400112 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08230dce-97a1-4b34-9018-c46c2af6881d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:45.402175 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-08230dce-97a1-4b34-9018-c46c2af6881d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:45.458011 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1378/2733] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:45.459654 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9ddb9-b9a4-478a-a9f2-38dd472e0135 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:45.459945 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9ddb9-b9a4-478a-a9f2-38dd472e0135 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:45.460223 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea9ddb9-b9a4-478a-a9f2-38dd472e0135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:45.495822 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-2ea9ddb9-b9a4-478a-a9f2-38dd472e0135 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:45.497494 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-2ea9ddb9-b9a4-478a-a9f2-38dd472e0135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:45.508534 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08230dce-97a1-4b34-9018-c46c2af6881d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:45.510155 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08230dce-97a1-4b34-9018-c46c2af6881d service cinder] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:45.517792 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1379/2734] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:45.522793 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea9ddb9-b9a4-478a-a9f2-38dd472e0135 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:45.531080 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1356/2735] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:31:45.535283 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea9ddb9-b9a4-478a-a9f2-38dd472e0135 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:45.541037 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a57e305-f295-40fc-9cb5-d1533fa03dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:45.542257 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a57e305-f295-40fc-9cb5-d1533fa03dbc admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:45.542634 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a57e305-f295-40fc-9cb5-d1533fa03dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:45.543074 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a57e305-f295-40fc-9cb5-d1533fa03dbc admin admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:45.544248 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a57e305-f295-40fc-9cb5-d1533fa03dbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:31:45.544664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a57e305-f295-40fc-9cb5-d1533fa03dbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:45.546648 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a57e305-f295-40fc-9cb5-d1533fa03dbc admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:45.569521 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1380/2736] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:31:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:31:45.569912 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08230dce-97a1-4b34-9018-c46c2af6881d service cinder] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:45.585751 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08230dce-97a1-4b34-9018-c46c2af6881d service cinder] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:45.602840 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f49b4-6a86-4c37-a466-29deef838386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:45.603614 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f49b4-6a86-4c37-a466-29deef838386 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:45.603953 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f49b4-6a86-4c37-a466-29deef838386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:45.604515 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f49b4-6a86-4c37-a466-29deef838386 admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:45.604929 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49f49b4-6a86-4c37-a466-29deef838386 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:45.606918 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49f49b4-6a86-4c37-a466-29deef838386 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:45.691594 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1357/2737] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:31:45 2022] POST /identity/v3/projects => generated 383 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:31:45.696259 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a57e305-f295-40fc-9cb5-d1533fa03dbc admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:45.713389 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a57e305-f295-40fc-9cb5-d1533fa03dbc admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:45.850476 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a077e927-aee9-46d9-95f9-1bb46f7da5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:45.850855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077e927-aee9-46d9-95f9-1bb46f7da5a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:45.851141 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077e927-aee9-46d9-95f9-1bb46f7da5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:45.851559 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a077e927-aee9-46d9-95f9-1bb46f7da5a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:45.852347 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a077e927-aee9-46d9-95f9-1bb46f7da5a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:45.855902 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a077e927-aee9-46d9-95f9-1bb46f7da5a6 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:45.919579 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1381/2738] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:31:45 2022] POST /identity/v3/users => generated 292 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:45.920242 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f49b4-6a86-4c37-a466-29deef838386 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:45.933615 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f49b4-6a86-4c37-a466-29deef838386 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:45.949420 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7211bb-6752-4f78-85ad-1346c4ba4397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:45.950293 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7211bb-6752-4f78-85ad-1346c4ba4397 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:45.950627 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7211bb-6752-4f78-85ad-1346c4ba4397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:45.950845 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7211bb-6752-4f78-85ad-1346c4ba4397 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:45.951591 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7211bb-6752-4f78-85ad-1346c4ba4397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:45.953315 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7211bb-6752-4f78-85ad-1346c4ba4397 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:45.988826 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a077e927-aee9-46d9-95f9-1bb46f7da5a6 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:45.992748 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1358/2739] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:45 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:46.006323 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a077e927-aee9-46d9-95f9-1bb46f7da5a6 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:46.013983 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b96bea-9838-46b4-b28b-9aa099dc917c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:46.015114 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b96bea-9838-46b4-b28b-9aa099dc917c admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:46.015697 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b96bea-9838-46b4-b28b-9aa099dc917c admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:46.016152 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b96bea-9838-46b4-b28b-9aa099dc917c admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:46.017205 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b96bea-9838-46b4-b28b-9aa099dc917c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:46.019219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b96bea-9838-46b4-b28b-9aa099dc917c admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:46.062995 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1382/2740] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:45 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:46.067137 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7211bb-6752-4f78-85ad-1346c4ba4397 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:46.082200 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7211bb-6752-4f78-85ad-1346c4ba4397 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:46.113472 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f937488-3581-443b-95f7-e88174996db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:46.114540 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f937488-3581-443b-95f7-e88174996db2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:46.114925 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f937488-3581-443b-95f7-e88174996db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:46.115310 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f937488-3581-443b-95f7-e88174996db2 admin admin] PATH_INFO: `/v3/projects/214b90db8a13452299de9352d73ba3bd/users/67a796e35b94452ebb1c7be98e9b327f/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:46.145141 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f937488-3581-443b-95f7-e88174996db2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=214b90db8a13452299de9352d73ba3bd, user_id=67a796e35b94452ebb1c7be98e9b327f, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:46.150566 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f937488-3581-443b-95f7-e88174996db2 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:46.185868 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1359/2741] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:31:46 2022] PUT /identity/v3/projects/214b90db8a13452299de9352d73ba3bd/users/67a796e35b94452ebb1c7be98e9b327f/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:46.197012 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab58dc-a4a9-4453-91e7-02111dd4140e None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:46.197587 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab58dc-a4a9-4453-91e7-02111dd4140e None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:46.197587 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab58dc-a4a9-4453-91e7-02111dd4140e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:46.254233 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-36ab58dc-a4a9-4453-91e7-02111dd4140e None None] MFA Rules not processed for user `67a796e35b94452ebb1c7be98e9b327f`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:46.256062 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-36ab58dc-a4a9-4453-91e7-02111dd4140e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:46.262403 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:46.408050 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1383/2742] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:46.438761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f937488-3581-443b-95f7-e88174996db2 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:46.441110 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-3f937488-3581-443b-95f7-e88174996db2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:46.470462 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:46.538121 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f937488-3581-443b-95f7-e88174996db2 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:46.549020 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefa0ac3-6705-4824-82b2-c4d87a854011 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:46.549829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa0ac3-6705-4824-82b2-c4d87a854011 service cinder] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:46.550745 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa0ac3-6705-4824-82b2-c4d87a854011 service cinder] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:46.550745 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa0ac3-6705-4824-82b2-c4d87a854011 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:46.562549 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefa0ac3-6705-4824-82b2-c4d87a854011 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:46.564290 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefa0ac3-6705-4824-82b2-c4d87a854011 service cinder] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:46.575396 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1360/2743] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:46.597966 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ab58dc-a4a9-4453-91e7-02111dd4140e None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:46.722489 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefa0ac3-6705-4824-82b2-c4d87a854011 service cinder] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:46.725163 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-cefa0ac3-6705-4824-82b2-c4d87a854011 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:46.764598 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ab58dc-a4a9-4453-91e7-02111dd4140e None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:46.767862 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefa0ac3-6705-4824-82b2-c4d87a854011 service cinder] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:46.769971 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b814975-d0ff-407b-b66c-cee96689494a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:46.771623 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b814975-d0ff-407b-b66c-cee96689494a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:46.772015 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b814975-d0ff-407b-b66c-cee96689494a admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:46.772849 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b814975-d0ff-407b-b66c-cee96689494a admin admin] PATH_INFO: `/v3/users/67a796e35b94452ebb1c7be98e9b327f` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:46.773603 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b814975-d0ff-407b-b66c-cee96689494a admin admin] RBAC: Authorizing `identity:delete_user(user_id=67a796e35b94452ebb1c7be98e9b327f)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:46.775514 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9fdd8b-d29a-492a-9c72-92b4991a6181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:46.775752 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b814975-d0ff-407b-b66c-cee96689494a admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:46.776410 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9fdd8b-d29a-492a-9c72-92b4991a6181 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:46.776689 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9fdd8b-d29a-492a-9c72-92b4991a6181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:46.777015 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9fdd8b-d29a-492a-9c72-92b4991a6181 admin admin] PATH_INFO: `/v3/users/e38e8354a69642fba0c0949fe5f5dab3` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:46.779097 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9fdd8b-d29a-492a-9c72-92b4991a6181 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e38e8354a69642fba0c0949fe5f5dab3)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:46.781123 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9fdd8b-d29a-492a-9c72-92b4991a6181 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:46.850751 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-ef9fdd8b-d29a-492a-9c72-92b4991a6181 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e38e8354a69642fba0c0949fe5f5dab3'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:46.868376 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-ef9fdd8b-d29a-492a-9c72-92b4991a6181 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e38e8354a69642fba0c0949fe5f5dab3'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:46.876570 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-9b814975-d0ff-407b-b66c-cee96689494a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67a796e35b94452ebb1c7be98e9b327f'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:46.877842 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-9b814975-d0ff-407b-b66c-cee96689494a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67a796e35b94452ebb1c7be98e9b327f'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:46.880204 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-ef9fdd8b-d29a-492a-9c72-92b4991a6181 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e38e8354a69642fba0c0949fe5f5dab3'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:46.891464 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-9b814975-d0ff-407b-b66c-cee96689494a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67a796e35b94452ebb1c7be98e9b327f'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:46.891846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-ef9fdd8b-d29a-492a-9c72-92b4991a6181 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e38e8354a69642fba0c0949fe5f5dab3'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:46.892550 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-ef9fdd8b-d29a-492a-9c72-92b4991a6181 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e38e8354a69642fba0c0949fe5f5dab3'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:46.892772 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-9b814975-d0ff-407b-b66c-cee96689494a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67a796e35b94452ebb1c7be98e9b327f'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:46.902896 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1361/2744] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:31:46 2022] DELETE /identity/v3/users/e38e8354a69642fba0c0949fe5f5dab3 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:46.926025 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-9b814975-d0ff-407b-b66c-cee96689494a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67a796e35b94452ebb1c7be98e9b327f'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:46.949401 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1384/2745] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:31:46 2022] DELETE /identity/v3/users/67a796e35b94452ebb1c7be98e9b327f => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:46.953091 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de1556-69df-461d-ae4f-64f45ef71bbf None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:46.953648 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de1556-69df-461d-ae4f-64f45ef71bbf None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:46.954124 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8de1556-69df-461d-ae4f-64f45ef71bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:46.992829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-f8de1556-69df-461d-ae4f-64f45ef71bbf None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:46.996523 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-f8de1556-69df-461d-ae4f-64f45ef71bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:47.096070 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1362/2746] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:47.106581 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b814975-d0ff-407b-b66c-cee96689494a admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:47.108562 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-9b814975-d0ff-407b-b66c-cee96689494a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:47.149865 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8de1556-69df-461d-ae4f-64f45ef71bbf None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:47.151589 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-f8de1556-69df-461d-ae4f-64f45ef71bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:47.175757 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:47.184646 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8de1556-69df-461d-ae4f-64f45ef71bbf None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:47.209681 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b24a488-14b7-45f9-91fc-991768692753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:47.210609 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24a488-14b7-45f9-91fc-991768692753 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:47.210914 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24a488-14b7-45f9-91fc-991768692753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:47.211198 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b24a488-14b7-45f9-91fc-991768692753 admin admin] PATH_INFO: `/v3/projects/f03723b938b845309929162c81adfe51` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:47.217376 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b24a488-14b7-45f9-91fc-991768692753 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f03723b938b845309929162c81adfe51)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:47.219409 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b24a488-14b7-45f9-91fc-991768692753 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:47.245914 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b814975-d0ff-407b-b66c-cee96689494a admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:47.251476 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fac79d-0df5-4f4a-8086-008e98e05379 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:47.252412 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fac79d-0df5-4f4a-8086-008e98e05379 service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:47.252924 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fac79d-0df5-4f4a-8086-008e98e05379 service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:47.253430 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fac79d-0df5-4f4a-8086-008e98e05379 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:47.265442 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fac79d-0df5-4f4a-8086-008e98e05379 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:47.267380 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fac79d-0df5-4f4a-8086-008e98e05379 service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:47.279989 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1385/2747] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:31:47.288327 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:47.346559 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-1b24a488-14b7-45f9-91fc-991768692753 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f03723b938b845309929162c81adfe51', 'request_id': 'req-1b24a488-14b7-45f9-91fc-991768692753'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:47.361212 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-1b24a488-14b7-45f9-91fc-991768692753 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f03723b938b845309929162c81adfe51', 'request_id': 'req-1b24a488-14b7-45f9-91fc-991768692753'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:47.368868 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-1b24a488-14b7-45f9-91fc-991768692753 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f03723b938b845309929162c81adfe51', 'request_id': 'req-1b24a488-14b7-45f9-91fc-991768692753'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:47.399262 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-1b24a488-14b7-45f9-91fc-991768692753 admin admin] The token cache is being invalidate because project f03723b938b845309929162c81adfe51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:31:47.399812 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-1b24a488-14b7-45f9-91fc-991768692753 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:47.401434 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-1b24a488-14b7-45f9-91fc-991768692753 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:47.406571 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1363/2748] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:31:47 2022] DELETE /identity/v3/projects/f03723b938b845309929162c81adfe51 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:31:47.408262 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fac79d-0df5-4f4a-8086-008e98e05379 service neutron] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:47.423223 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fac79d-0df5-4f4a-8086-008e98e05379 service neutron] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:47.535084 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b5c4b-0976-4f51-8640-738a6bea0e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:47.536277 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b5c4b-0976-4f51-8640-738a6bea0e87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:47.536560 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b5c4b-0976-4f51-8640-738a6bea0e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:47.536837 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b5c4b-0976-4f51-8640-738a6bea0e87 admin admin] PATH_INFO: `/v3/users/14aa67984a0d468d84c5f58c6b0b867f` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:47.538247 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3b5c4b-0976-4f51-8640-738a6bea0e87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14aa67984a0d468d84c5f58c6b0b867f)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:47.541053 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3b5c4b-0976-4f51-8640-738a6bea0e87 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:47.595911 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b24a488-14b7-45f9-91fc-991768692753 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:47.597930 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-1b24a488-14b7-45f9-91fc-991768692753 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:47.646112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-5d3b5c4b-0976-4f51-8640-738a6bea0e87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14aa67984a0d468d84c5f58c6b0b867f'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:47.646932 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-5d3b5c4b-0976-4f51-8640-738a6bea0e87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14aa67984a0d468d84c5f58c6b0b867f'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:47.668558 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-5d3b5c4b-0976-4f51-8640-738a6bea0e87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14aa67984a0d468d84c5f58c6b0b867f'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:47.669756 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-5d3b5c4b-0976-4f51-8640-738a6bea0e87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14aa67984a0d468d84c5f58c6b0b867f'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:47.696720 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-5d3b5c4b-0976-4f51-8640-738a6bea0e87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14aa67984a0d468d84c5f58c6b0b867f'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:47.729977 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1386/2749] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:31:47 2022] DELETE /identity/v3/users/14aa67984a0d468d84c5f58c6b0b867f => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:47.736386 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b24a488-14b7-45f9-91fc-991768692753 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:47.792188 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7405a-ca81-416b-974c-58e9c2777c3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:47.792837 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7405a-ca81-416b-974c-58e9c2777c3e service cinder] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:47.793430 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7405a-ca81-416b-974c-58e9c2777c3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:47.793430 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7405a-ca81-416b-974c-58e9c2777c3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:47.795649 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-00d7405a-ca81-416b-974c-58e9c2777c3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:47.826089 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b5c4b-0976-4f51-8640-738a6bea0e87 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:47.828245 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-5d3b5c4b-0976-4f51-8640-738a6bea0e87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:47.924767 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d7405a-ca81-416b-974c-58e9c2777c3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:47.925454 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:47.926407 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d7405a-ca81-416b-974c-58e9c2777c3e service cinder] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:47.946607 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b5c4b-0976-4f51-8640-738a6bea0e87 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:47.948227 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1364/2750] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:31:47.956126 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f97974-a060-4a84-847b-13bd3f426e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:47.957322 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f97974-a060-4a84-847b-13bd3f426e79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:47.957873 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f97974-a060-4a84-847b-13bd3f426e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:47.958328 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f97974-a060-4a84-847b-13bd3f426e79 admin admin] PATH_INFO: `/v3/projects/214b90db8a13452299de9352d73ba3bd` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:47.961847 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f97974-a060-4a84-847b-13bd3f426e79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=214b90db8a13452299de9352d73ba3bd)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:47.962534 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f97974-a060-4a84-847b-13bd3f426e79 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:47.985250 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7405a-ca81-416b-974c-58e9c2777c3e service cinder] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:47.986781 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-00d7405a-ca81-416b-974c-58e9c2777c3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:48.017183 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7405a-ca81-416b-974c-58e9c2777c3e service cinder] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:48.019164 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-05f97974-a060-4a84-847b-13bd3f426e79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '214b90db8a13452299de9352d73ba3bd', 'request_id': 'req-05f97974-a060-4a84-847b-13bd3f426e79'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:48.026956 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-05f97974-a060-4a84-847b-13bd3f426e79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '214b90db8a13452299de9352d73ba3bd', 'request_id': 'req-05f97974-a060-4a84-847b-13bd3f426e79'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:48.037120 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-05f97974-a060-4a84-847b-13bd3f426e79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '214b90db8a13452299de9352d73ba3bd', 'request_id': 'req-05f97974-a060-4a84-847b-13bd3f426e79'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:48.040036 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:48.052448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-05f97974-a060-4a84-847b-13bd3f426e79 admin admin] The token cache is being invalidate because project 214b90db8a13452299de9352d73ba3bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:31:48.052866 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-05f97974-a060-4a84-847b-13bd3f426e79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:48.053855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-05f97974-a060-4a84-847b-13bd3f426e79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:48.056164 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1387/2751] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:31:47 2022] DELETE /identity/v3/projects/214b90db8a13452299de9352d73ba3bd => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:48.061929 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3ffce9-8df7-4d7d-bca2-f5c96f22971a None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:48.062217 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3ffce9-8df7-4d7d-bca2-f5c96f22971a None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:48.062568 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3ffce9-8df7-4d7d-bca2-f5c96f22971a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:48.083248 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-3e3ffce9-8df7-4d7d-bca2-f5c96f22971a None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:48.084601 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-3e3ffce9-8df7-4d7d-bca2-f5c96f22971a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:48.088769 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458f05e-5fcc-4823-9818-fd5a72967ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:48.089711 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458f05e-5fcc-4823-9818-fd5a72967ee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:48.090142 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458f05e-5fcc-4823-9818-fd5a72967ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:48.090438 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458f05e-5fcc-4823-9818-fd5a72967ee6 admin admin] PATH_INFO: `/v3/projects/afbd09946f09447a92a186270aa81079` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:48.093437 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4458f05e-5fcc-4823-9818-fd5a72967ee6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afbd09946f09447a92a186270aa81079)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:48.094665 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1388/2752] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:31:48.095674 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4458f05e-5fcc-4823-9818-fd5a72967ee6 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:48.102443 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a2e50-d6d7-4723-a693-1570208196ec None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:48.102973 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a2e50-d6d7-4723-a693-1570208196ec None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:48.103407 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0a2e50-d6d7-4723-a693-1570208196ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:48.124505 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-ee0a2e50-d6d7-4723-a693-1570208196ec None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:48.125717 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-ee0a2e50-d6d7-4723-a693-1570208196ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:48.139497 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1389/2753] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:48.144228 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a2e50-d6d7-4723-a693-1570208196ec None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:48.154484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0a2e50-d6d7-4723-a693-1570208196ec None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:48.155136 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-4458f05e-5fcc-4823-9818-fd5a72967ee6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afbd09946f09447a92a186270aa81079', 'request_id': 'req-4458f05e-5fcc-4823-9818-fd5a72967ee6'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:48.162000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-4458f05e-5fcc-4823-9818-fd5a72967ee6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afbd09946f09447a92a186270aa81079', 'request_id': 'req-4458f05e-5fcc-4823-9818-fd5a72967ee6'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:48.168300 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-4458f05e-5fcc-4823-9818-fd5a72967ee6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afbd09946f09447a92a186270aa81079', 'request_id': 'req-4458f05e-5fcc-4823-9818-fd5a72967ee6'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:48.185221 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-4458f05e-5fcc-4823-9818-fd5a72967ee6 admin admin] The token cache is being invalidate because project afbd09946f09447a92a186270aa81079 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:31:48.185514 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-4458f05e-5fcc-4823-9818-fd5a72967ee6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:48.186231 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-4458f05e-5fcc-4823-9818-fd5a72967ee6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:48.200823 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1365/2754] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:31:47 2022] DELETE /identity/v3/projects/afbd09946f09447a92a186270aa81079 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:48.222998 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862819f0-e80b-4fc9-8a08-9055d4cbee3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:48.223788 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862819f0-e80b-4fc9-8a08-9055d4cbee3f admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:48.224132 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862819f0-e80b-4fc9-8a08-9055d4cbee3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:48.224489 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862819f0-e80b-4fc9-8a08-9055d4cbee3f admin admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:48.225339 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862819f0-e80b-4fc9-8a08-9055d4cbee3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:31:48.225719 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862819f0-e80b-4fc9-8a08-9055d4cbee3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:48.227495 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862819f0-e80b-4fc9-8a08-9055d4cbee3f admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:48.243900 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1390/2755] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:31:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:31:48.248729 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458f05e-5fcc-4823-9818-fd5a72967ee6 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:48.250556 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-4458f05e-5fcc-4823-9818-fd5a72967ee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:48.266626 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458f05e-5fcc-4823-9818-fd5a72967ee6 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:48.273220 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06524a1-f916-41a8-9234-633c6d0a05cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:48.274043 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06524a1-f916-41a8-9234-633c6d0a05cc admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:48.274481 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06524a1-f916-41a8-9234-633c6d0a05cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:48.274863 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06524a1-f916-41a8-9234-633c6d0a05cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:48.278197 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06524a1-f916-41a8-9234-633c6d0a05cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:48.280841 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06524a1-f916-41a8-9234-633c6d0a05cc admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:48.316632 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1366/2756] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:31:48 2022] POST /identity/v3/projects => generated 375 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:48.320941 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862819f0-e80b-4fc9-8a08-9055d4cbee3f admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:48.332474 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862819f0-e80b-4fc9-8a08-9055d4cbee3f admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:48.414876 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:48.422918 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a2fdc8-611a-4955-b46b-e616a3c53413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:48.424613 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2fdc8-611a-4955-b46b-e616a3c53413 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:48.425151 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2fdc8-611a-4955-b46b-e616a3c53413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:48.426340 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2fdc8-611a-4955-b46b-e616a3c53413 admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:48.427432 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a2fdc8-611a-4955-b46b-e616a3c53413 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:48.430033 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a2fdc8-611a-4955-b46b-e616a3c53413 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:48.464149 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1391/2757] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:31:48 2022] POST /identity/v3/users => generated 288 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:48.464311 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06524a1-f916-41a8-9234-633c6d0a05cc admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:48.480473 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06524a1-f916-41a8-9234-633c6d0a05cc admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:48.492010 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e94e6-0105-4d52-93bf-8859a06c8cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:48.492777 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e94e6-0105-4d52-93bf-8859a06c8cab admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:48.493153 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e94e6-0105-4d52-93bf-8859a06c8cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:48.493491 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e94e6-0105-4d52-93bf-8859a06c8cab admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:48.494230 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291e94e6-0105-4d52-93bf-8859a06c8cab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:48.495920 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291e94e6-0105-4d52-93bf-8859a06c8cab admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:48.507062 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1367/2758] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:48.522151 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a2fdc8-611a-4955-b46b-e616a3c53413 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:48.524543 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b0bee-b3f2-4cad-9586-15f999121f25 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:48.524833 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b0bee-b3f2-4cad-9586-15f999121f25 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:48.525111 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b0bee-b3f2-4cad-9586-15f999121f25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:48.533379 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a2fdc8-611a-4955-b46b-e616a3c53413 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:48.538142 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198af98-13a8-488f-80ce-810acd10c2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:48.538767 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198af98-13a8-488f-80ce-810acd10c2c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:48.539052 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198af98-13a8-488f-80ce-810acd10c2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:48.539325 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7198af98-13a8-488f-80ce-810acd10c2c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:48.540049 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7198af98-13a8-488f-80ce-810acd10c2c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:48.541693 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7198af98-13a8-488f-80ce-810acd10c2c3 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:48.556059 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1392/2759] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:48.556377 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-b21b0bee-b3f2-4cad-9586-15f999121f25 None None] MFA Rules not processed for user `86a18e313e4944748aef58d54bfe7b13`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:48.556377 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-b21b0bee-b3f2-4cad-9586-15f999121f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:48.562605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198af98-13a8-488f-80ce-810acd10c2c3 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:48.573133 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7198af98-13a8-488f-80ce-810acd10c2c3 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:48.578792 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453a39e8-e5eb-4579-8003-17b219a97191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:48.579687 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a39e8-e5eb-4579-8003-17b219a97191 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:48.579966 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a39e8-e5eb-4579-8003-17b219a97191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:48.580241 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453a39e8-e5eb-4579-8003-17b219a97191 admin admin] PATH_INFO: `/v3/projects/868f8959de474408b53b31d7cbc49325/users/ae63e5c5006640fdbfdbb90ce6f404f7/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:48.597855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a39e8-e5eb-4579-8003-17b219a97191 admin admin] RBAC: Authorizing `identity:create_grant(project_id=868f8959de474408b53b31d7cbc49325, user_id=ae63e5c5006640fdbfdbb90ce6f404f7, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:48.600699 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453a39e8-e5eb-4579-8003-17b219a97191 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:48.618878 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1393/2760] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:31:48 2022] PUT /identity/v3/projects/868f8959de474408b53b31d7cbc49325/users/ae63e5c5006640fdbfdbb90ce6f404f7/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:48.623054 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1368/2761] 10.222.0.50 () {58 vars in 967 bytes} [Fri May 27 03:31:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:48.634613 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad73f24-5043-45d7-b898-416315e50bb6 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:48.634906 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad73f24-5043-45d7-b898-416315e50bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:48.635225 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad73f24-5043-45d7-b898-416315e50bb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:48.642645 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b0bee-b3f2-4cad-9586-15f999121f25 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:48.644137 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-b21b0bee-b3f2-4cad-9586-15f999121f25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:48.668912 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-aad73f24-5043-45d7-b898-416315e50bb6 None None] MFA Rules not processed for user `ae63e5c5006640fdbfdbb90ce6f404f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:48.670141 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-aad73f24-5043-45d7-b898-416315e50bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:48.674450 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:48.729353 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b0bee-b3f2-4cad-9586-15f999121f25 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:48.745040 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1394/2762] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:48.745898 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad73f24-5043-45d7-b898-416315e50bb6 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:48.747564 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-aad73f24-5043-45d7-b898-416315e50bb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:48.793934 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8cb0f4-f6a0-4765-882f-f4c1aa05e64f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de53e2f0fba4e0fac70887067c1963d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:48.794600 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8cb0f4-f6a0-4765-882f-f4c1aa05e64f service swift] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:48.794882 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8cb0f4-f6a0-4765-882f-f4c1aa05e64f service swift] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:48.795155 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8cb0f4-f6a0-4765-882f-f4c1aa05e64f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:48.803306 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad73f24-5043-45d7-b898-416315e50bb6 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:48.806956 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ae665-911a-49e8-a8a6-265a4e6a9d78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:48.807602 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ae665-911a-49e8-a8a6-265a4e6a9d78 service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:48.807878 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ae665-911a-49e8-a8a6-265a4e6a9d78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:48.808152 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ae665-911a-49e8-a8a6-265a4e6a9d78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:48.810052 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-bc9ae665-911a-49e8-a8a6-265a4e6a9d78 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:48.854217 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8cb0f4-f6a0-4765-882f-f4c1aa05e64f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:48.855863 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8cb0f4-f6a0-4765-882f-f4c1aa05e64f service swift] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:48.863336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:48.864393 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1369/2763] 10.222.0.50 () {60 vars in 1384 bytes} [Fri May 27 03:31:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 27 03:31:48.865079 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8cb0f4-f6a0-4765-882f-f4c1aa05e64f service swift] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:48.866778 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-bc8cb0f4-f6a0-4765-882f-f4c1aa05e64f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:48.877392 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ae665-911a-49e8-a8a6-265a4e6a9d78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:48.879027 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ae665-911a-49e8-a8a6-265a4e6a9d78 service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:48.890837 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1395/2764] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:48 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:48.891931 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ae665-911a-49e8-a8a6-265a4e6a9d78 service cinder] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:48.959429 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8cb0f4-f6a0-4765-882f-f4c1aa05e64f service swift] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:48.965379 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ae665-911a-49e8-a8a6-265a4e6a9d78 service cinder] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:48.972884 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee5f00d-439d-4da3-ac4d-00b2920d7068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:48.974423 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5e3ca-6c0b-4295-b0e3-50c26ff94d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:48.974568 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5f00d-439d-4da3-ac4d-00b2920d7068 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:48.975540 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5e3ca-6c0b-4295-b0e3-50c26ff94d11 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:48.976081 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5e3ca-6c0b-4295-b0e3-50c26ff94d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:48.976529 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef5e3ca-6c0b-4295-b0e3-50c26ff94d11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:48.976756 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5f00d-439d-4da3-ac4d-00b2920d7068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:48.977212 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5f00d-439d-4da3-ac4d-00b2920d7068 admin admin] PATH_INFO: `/v3/users/4d1433ea32cd4b809ff71c787e6a26e4` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:48.977844 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef5e3ca-6c0b-4295-b0e3-50c26ff94d11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:48.979154 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee5f00d-439d-4da3-ac4d-00b2920d7068 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d1433ea32cd4b809ff71c787e6a26e4)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:48.982209 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef5e3ca-6c0b-4295-b0e3-50c26ff94d11 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:48.982528 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee5f00d-439d-4da3-ac4d-00b2920d7068 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:49.002747 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:49.019218 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1396/2765] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:31:48 2022] POST /identity/v3/projects => generated 375 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:49.023362 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5e3ca-6c0b-4295-b0e3-50c26ff94d11 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:49.070156 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-fee5f00d-439d-4da3-ac4d-00b2920d7068 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d1433ea32cd4b809ff71c787e6a26e4'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:49.089010 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-fee5f00d-439d-4da3-ac4d-00b2920d7068 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d1433ea32cd4b809ff71c787e6a26e4'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:49.096162 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef5e3ca-6c0b-4295-b0e3-50c26ff94d11 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:49.100363 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-fee5f00d-439d-4da3-ac4d-00b2920d7068 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d1433ea32cd4b809ff71c787e6a26e4'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:49.101275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe1fc2-21f1-41aa-9f08-7e311fa545f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:49.102031 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe1fc2-21f1-41aa-9f08-7e311fa545f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:49.102401 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe1fc2-21f1-41aa-9f08-7e311fa545f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:49.102684 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe1fc2-21f1-41aa-9f08-7e311fa545f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:49.103390 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbe1fc2-21f1-41aa-9f08-7e311fa545f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:49.105415 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbe1fc2-21f1-41aa-9f08-7e311fa545f1 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:49.112339 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-fee5f00d-439d-4da3-ac4d-00b2920d7068 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d1433ea32cd4b809ff71c787e6a26e4'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:49.114077 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-fee5f00d-439d-4da3-ac4d-00b2920d7068 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d1433ea32cd4b809ff71c787e6a26e4'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:49.120898 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1370/2766] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:31:48 2022] DELETE /identity/v3/users/4d1433ea32cd4b809ff71c787e6a26e4 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:49.126352 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e955c-9ee9-4ba5-a08e-1cc90f27f0f6 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:49.126813 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e955c-9ee9-4ba5-a08e-1cc90f27f0f6 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:49.127150 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e955c-9ee9-4ba5-a08e-1cc90f27f0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:49.130672 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1397/2767] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:31:49 2022] POST /identity/v3/users => generated 294 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:31:49.136535 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe1fc2-21f1-41aa-9f08-7e311fa545f1 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:49.138958 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-6bbe1fc2-21f1-41aa-9f08-7e311fa545f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:49.151466 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-b88e955c-9ee9-4ba5-a08e-1cc90f27f0f6 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:49.152739 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe1fc2-21f1-41aa-9f08-7e311fa545f1 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:49.152911 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-b88e955c-9ee9-4ba5-a08e-1cc90f27f0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:49.218798 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae4653e-a54e-441f-8ab1-9d0eb9a8b00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:49.219506 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4653e-a54e-441f-8ab1-9d0eb9a8b00d admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:49.219789 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4653e-a54e-441f-8ab1-9d0eb9a8b00d admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:49.220062 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae4653e-a54e-441f-8ab1-9d0eb9a8b00d admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:49.220928 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae4653e-a54e-441f-8ab1-9d0eb9a8b00d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:49.222605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae4653e-a54e-441f-8ab1-9d0eb9a8b00d admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:49.227052 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1371/2768] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:31:49.234781 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1398/2769] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:49.239410 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e955c-9ee9-4ba5-a08e-1cc90f27f0f6 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:49.239932 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae4653e-a54e-441f-8ab1-9d0eb9a8b00d admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:49.242795 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-0ae4653e-a54e-441f-8ab1-9d0eb9a8b00d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:49.248882 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:49.251031 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e955c-9ee9-4ba5-a08e-1cc90f27f0f6 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:49.256634 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a10aa3-181d-427b-a0d5-5afe9f8bef8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:49.257662 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a10aa3-181d-427b-a0d5-5afe9f8bef8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:49.257950 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a10aa3-181d-427b-a0d5-5afe9f8bef8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:49.258224 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a10aa3-181d-427b-a0d5-5afe9f8bef8c admin admin] PATH_INFO: `/v3/projects/f986b88547aa40eb8f0fc5ba2594afea/users/8a753b1bfac849be9e647b378462e917/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:49.285689 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a10aa3-181d-427b-a0d5-5afe9f8bef8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f986b88547aa40eb8f0fc5ba2594afea, user_id=8a753b1bfac849be9e647b378462e917, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:49.290434 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a10aa3-181d-427b-a0d5-5afe9f8bef8c admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:49.310301 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1372/2770] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:31:49 2022] PUT /identity/v3/projects/f986b88547aa40eb8f0fc5ba2594afea/users/8a753b1bfac849be9e647b378462e917/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:31:49.314301 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a10aa3-181d-427b-a0d5-5afe9f8bef8c admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:49.323853 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae4653e-a54e-441f-8ab1-9d0eb9a8b00d admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:49.385442 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654a64f6-3b01-4d92-8e0e-fa74ce83655f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:49.386098 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a64f6-3b01-4d92-8e0e-fa74ce83655f service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:49.386371 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a64f6-3b01-4d92-8e0e-fa74ce83655f service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:49.386634 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654a64f6-3b01-4d92-8e0e-fa74ce83655f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:49.394525 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a10aa3-181d-427b-a0d5-5afe9f8bef8c admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:49.396292 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654a64f6-3b01-4d92-8e0e-fa74ce83655f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:49.397946 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654a64f6-3b01-4d92-8e0e-fa74ce83655f service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:49.399088 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919d418b-870f-445e-abee-dc1ec419254c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:49.399755 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d418b-870f-445e-abee-dc1ec419254c admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:49.400036 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d418b-870f-445e-abee-dc1ec419254c admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:49.400302 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d418b-870f-445e-abee-dc1ec419254c admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:49.401078 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919d418b-870f-445e-abee-dc1ec419254c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:49.402846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919d418b-870f-445e-abee-dc1ec419254c admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:49.405601 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:49.408175 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1399/2771] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:49.417736 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1373/2772] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:49.419230 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654a64f6-3b01-4d92-8e0e-fa74ce83655f service neutron] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:49.429175 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654a64f6-3b01-4d92-8e0e-fa74ce83655f service neutron] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:49.433869 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb578ae5-cbfc-486a-9345-bb5275a28392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:49.434838 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb578ae5-cbfc-486a-9345-bb5275a28392 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:49.435141 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb578ae5-cbfc-486a-9345-bb5275a28392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:49.435408 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb578ae5-cbfc-486a-9345-bb5275a28392 admin admin] PATH_INFO: `/v3/projects/f986b88547aa40eb8f0fc5ba2594afea/users/8a753b1bfac849be9e647b378462e917/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:49.437405 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb578ae5-cbfc-486a-9345-bb5275a28392 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f986b88547aa40eb8f0fc5ba2594afea, user_id=8a753b1bfac849be9e647b378462e917, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:49.440039 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb578ae5-cbfc-486a-9345-bb5275a28392 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:49.451265 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1400/2773] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:31:49 2022] PUT /identity/v3/projects/f986b88547aa40eb8f0fc5ba2594afea/users/8a753b1bfac849be9e647b378462e917/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:49.462631 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2015c8cc-6db7-480c-a9d5-f2a16198b83d None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:49.463489 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2015c8cc-6db7-480c-a9d5-f2a16198b83d None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:49.463845 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2015c8cc-6db7-480c-a9d5-f2a16198b83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:49.492372 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-2015c8cc-6db7-480c-a9d5-f2a16198b83d None None] MFA Rules not processed for user `8a753b1bfac849be9e647b378462e917`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:49.493711 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-2015c8cc-6db7-480c-a9d5-f2a16198b83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:49.585377 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1374/2774] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:49 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:49.593859 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb578ae5-cbfc-486a-9345-bb5275a28392 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:49.596241 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-fb578ae5-cbfc-486a-9345-bb5275a28392 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:49.662324 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb578ae5-cbfc-486a-9345-bb5275a28392 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:49.665942 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a587999a-11bf-436c-a99d-2e62facb6a93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:49.667002 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587999a-11bf-436c-a99d-2e62facb6a93 service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:49.667002 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587999a-11bf-436c-a99d-2e62facb6a93 service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:49.667297 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587999a-11bf-436c-a99d-2e62facb6a93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:49.669422 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-a587999a-11bf-436c-a99d-2e62facb6a93 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:49.694770 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2015c8cc-6db7-480c-a9d5-f2a16198b83d None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:49.696289 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-2015c8cc-6db7-480c-a9d5-f2a16198b83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:49.742094 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a587999a-11bf-436c-a99d-2e62facb6a93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:49.744687 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a587999a-11bf-436c-a99d-2e62facb6a93 service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:49.759474 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1401/2775] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:49.776330 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2015c8cc-6db7-480c-a9d5-f2a16198b83d None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:49.780802 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ca078-db0a-4c52-89d1-7e3e72931653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:49.781827 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ca078-db0a-4c52-89d1-7e3e72931653 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:49.782105 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ca078-db0a-4c52-89d1-7e3e72931653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:49.782374 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ca078-db0a-4c52-89d1-7e3e72931653 admin admin] PATH_INFO: `/v3/projects/141475de7c9742fcb8c11fc56d620bb0` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:49.783539 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4ca078-db0a-4c52-89d1-7e3e72931653 admin admin] RBAC: Authorizing `identity:delete_project(project_id=141475de7c9742fcb8c11fc56d620bb0)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:49.785581 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4ca078-db0a-4c52-89d1-7e3e72931653 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:49.837923 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-1c4ca078-db0a-4c52-89d1-7e3e72931653 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '141475de7c9742fcb8c11fc56d620bb0', 'request_id': 'req-1c4ca078-db0a-4c52-89d1-7e3e72931653'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:49.845609 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-1c4ca078-db0a-4c52-89d1-7e3e72931653 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '141475de7c9742fcb8c11fc56d620bb0', 'request_id': 'req-1c4ca078-db0a-4c52-89d1-7e3e72931653'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:49.851240 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-1c4ca078-db0a-4c52-89d1-7e3e72931653 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '141475de7c9742fcb8c11fc56d620bb0', 'request_id': 'req-1c4ca078-db0a-4c52-89d1-7e3e72931653'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:49.865537 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-1c4ca078-db0a-4c52-89d1-7e3e72931653 admin admin] The token cache is being invalidate because project 141475de7c9742fcb8c11fc56d620bb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:31:49.865828 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-1c4ca078-db0a-4c52-89d1-7e3e72931653 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:49.866447 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-1c4ca078-db0a-4c52-89d1-7e3e72931653 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:49.867989 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1375/2776] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:31:49 2022] DELETE /identity/v3/projects/141475de7c9742fcb8c11fc56d620bb0 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:49.873276 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38b5f8-1558-4cfb-a5a5-5fe39da78382 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:49.873671 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38b5f8-1558-4cfb-a5a5-5fe39da78382 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:49.874030 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38b5f8-1558-4cfb-a5a5-5fe39da78382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:49.880568 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:49.896186 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-3a38b5f8-1558-4cfb-a5a5-5fe39da78382 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:49.898190 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-3a38b5f8-1558-4cfb-a5a5-5fe39da78382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:49.965314 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1402/2777] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:49.973462 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16545e02-14f0-4c46-8add-9abfeb7522d8 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:49.973854 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16545e02-14f0-4c46-8add-9abfeb7522d8 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:49.974192 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16545e02-14f0-4c46-8add-9abfeb7522d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:49.994666 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-16545e02-14f0-4c46-8add-9abfeb7522d8 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:49.995908 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-16545e02-14f0-4c46-8add-9abfeb7522d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:50.004443 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1376/2778] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:50.009645 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a38b5f8-1558-4cfb-a5a5-5fe39da78382 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:50.021027 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a38b5f8-1558-4cfb-a5a5-5fe39da78382 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:50.026113 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a9b714-fee2-450d-92e4-4f3978f7ce94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:50.028975 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9b714-fee2-450d-92e4-4f3978f7ce94 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:50.029347 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9b714-fee2-450d-92e4-4f3978f7ce94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:50.029780 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a9b714-fee2-450d-92e4-4f3978f7ce94 admin admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:50.030559 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a9b714-fee2-450d-92e4-4f3978f7ce94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:31:50.030942 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a9b714-fee2-450d-92e4-4f3978f7ce94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:50.032602 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a9b714-fee2-450d-92e4-4f3978f7ce94 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:50.046563 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1403/2779] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:31:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:31:50.051135 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16545e02-14f0-4c46-8add-9abfeb7522d8 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:50.066864 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16545e02-14f0-4c46-8add-9abfeb7522d8 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:50.073828 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81df038-1fec-456a-a647-fcc0724c4d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:50.074823 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81df038-1fec-456a-a647-fcc0724c4d3d admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:50.075109 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81df038-1fec-456a-a647-fcc0724c4d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:50.075370 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81df038-1fec-456a-a647-fcc0724c4d3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:50.076084 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81df038-1fec-456a-a647-fcc0724c4d3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:50.078065 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81df038-1fec-456a-a647-fcc0724c4d3d admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:50.088571 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:50.127184 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1377/2780] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:31:50 2022] POST /identity/v3/projects => generated 371 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:50.128472 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a9b714-fee2-450d-92e4-4f3978f7ce94 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:50.141664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a9b714-fee2-450d-92e4-4f3978f7ce94 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:50.210179 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f7c54d-59f9-4921-88d2-0992dae0e188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:50.210896 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f7c54d-59f9-4921-88d2-0992dae0e188 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:50.211176 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f7c54d-59f9-4921-88d2-0992dae0e188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:50.211457 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f7c54d-59f9-4921-88d2-0992dae0e188 admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:50.212161 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f7c54d-59f9-4921-88d2-0992dae0e188 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:50.214438 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f7c54d-59f9-4921-88d2-0992dae0e188 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:50.241313 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1404/2781] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:31:50 2022] POST /identity/v3/users => generated 286 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:50.243040 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81df038-1fec-456a-a647-fcc0724c4d3d admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:50.254438 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81df038-1fec-456a-a647-fcc0724c4d3d admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:50.259631 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a76c6d-edae-40f0-b7bb-4f1cbd7d8e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:50.260378 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a76c6d-edae-40f0-b7bb-4f1cbd7d8e14 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:50.260646 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a76c6d-edae-40f0-b7bb-4f1cbd7d8e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:50.260940 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a76c6d-edae-40f0-b7bb-4f1cbd7d8e14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:50.261727 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a76c6d-edae-40f0-b7bb-4f1cbd7d8e14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:50.263628 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a76c6d-edae-40f0-b7bb-4f1cbd7d8e14 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:50.276361 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1378/2782] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:50.281395 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f7c54d-59f9-4921-88d2-0992dae0e188 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:50.292621 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f7c54d-59f9-4921-88d2-0992dae0e188 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:50.298101 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c0bd60-4fe6-4e1e-922a-bb4806c4115c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:50.298919 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c0bd60-4fe6-4e1e-922a-bb4806c4115c admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:50.299206 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c0bd60-4fe6-4e1e-922a-bb4806c4115c admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:50.299541 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c0bd60-4fe6-4e1e-922a-bb4806c4115c admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:50.300275 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c0bd60-4fe6-4e1e-922a-bb4806c4115c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:50.301939 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c0bd60-4fe6-4e1e-922a-bb4806c4115c admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:50.315849 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1405/2783] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:50.320185 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a76c6d-edae-40f0-b7bb-4f1cbd7d8e14 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:50.331511 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a76c6d-edae-40f0-b7bb-4f1cbd7d8e14 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:50.337250 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a344413b-3ecd-4745-9a08-587da93d8374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:50.338177 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344413b-3ecd-4745-9a08-587da93d8374 admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:50.338451 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344413b-3ecd-4745-9a08-587da93d8374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:50.338722 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344413b-3ecd-4745-9a08-587da93d8374 admin admin] PATH_INFO: `/v3/projects/ec3c28c54bc041c088bbcc71521b0a4d/users/b1237ac3fecf4f578e00a1626af862e5/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:50.354858 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a344413b-3ecd-4745-9a08-587da93d8374 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec3c28c54bc041c088bbcc71521b0a4d, user_id=b1237ac3fecf4f578e00a1626af862e5, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:50.358299 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a344413b-3ecd-4745-9a08-587da93d8374 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:50.372649 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1379/2784] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:31:50 2022] PUT /identity/v3/projects/ec3c28c54bc041c088bbcc71521b0a4d/users/b1237ac3fecf4f578e00a1626af862e5/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:31:50.383855 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eba277-7fc8-4653-86f8-7bc07523f9e1 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:50.384367 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eba277-7fc8-4653-86f8-7bc07523f9e1 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:50.384466 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eba277-7fc8-4653-86f8-7bc07523f9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:50.412676 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-b9eba277-7fc8-4653-86f8-7bc07523f9e1 None None] MFA Rules not processed for user `b1237ac3fecf4f578e00a1626af862e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:50.413976 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-b9eba277-7fc8-4653-86f8-7bc07523f9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:50.482066 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:50.492490 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1406/2785] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:50.500084 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a344413b-3ecd-4745-9a08-587da93d8374 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:50.501953 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-a344413b-3ecd-4745-9a08-587da93d8374 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:50.561884 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a344413b-3ecd-4745-9a08-587da93d8374 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:50.565560 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584f442e-f23b-40f9-9ced-e05ef3118f22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:50.566210 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f442e-f23b-40f9-9ced-e05ef3118f22 service cinder] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:50.566493 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f442e-f23b-40f9-9ced-e05ef3118f22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:50.566805 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f442e-f23b-40f9-9ced-e05ef3118f22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:50.575731 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584f442e-f23b-40f9-9ced-e05ef3118f22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:50.577305 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584f442e-f23b-40f9-9ced-e05ef3118f22 service cinder] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:50.586491 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1380/2786] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:50.614786 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eba277-7fc8-4653-86f8-7bc07523f9e1 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:50.696137 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eba277-7fc8-4653-86f8-7bc07523f9e1 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:50.701460 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1869305-2e58-4b25-bedf-5bcd80e281fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:50.702344 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1869305-2e58-4b25-bedf-5bcd80e281fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:50.703251 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1869305-2e58-4b25-bedf-5bcd80e281fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:50.703532 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1869305-2e58-4b25-bedf-5bcd80e281fc admin admin] PATH_INFO: `/v3/users/b1237ac3fecf4f578e00a1626af862e5` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:50.704849 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1869305-2e58-4b25-bedf-5bcd80e281fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1237ac3fecf4f578e00a1626af862e5)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:50.706822 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1869305-2e58-4b25-bedf-5bcd80e281fc admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:50.755998 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-a1869305-2e58-4b25-bedf-5bcd80e281fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1237ac3fecf4f578e00a1626af862e5'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:50.756826 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-a1869305-2e58-4b25-bedf-5bcd80e281fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1237ac3fecf4f578e00a1626af862e5'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:50.766352 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-a1869305-2e58-4b25-bedf-5bcd80e281fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1237ac3fecf4f578e00a1626af862e5'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:50.767064 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-a1869305-2e58-4b25-bedf-5bcd80e281fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1237ac3fecf4f578e00a1626af862e5'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:50.781270 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-a1869305-2e58-4b25-bedf-5bcd80e281fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1237ac3fecf4f578e00a1626af862e5'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:50.794581 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1407/2787] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:31:50 2022] DELETE /identity/v3/users/b1237ac3fecf4f578e00a1626af862e5 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:50.799288 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68caab2c-5ff1-421b-8276-8cf0a439c936 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:50.799625 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68caab2c-5ff1-421b-8276-8cf0a439c936 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:50.799907 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68caab2c-5ff1-421b-8276-8cf0a439c936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:50.818534 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-68caab2c-5ff1-421b-8276-8cf0a439c936 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:50.819741 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-68caab2c-5ff1-421b-8276-8cf0a439c936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:50.886147 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1381/2788] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:50.896356 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1869305-2e58-4b25-bedf-5bcd80e281fc admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:50.898049 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-a1869305-2e58-4b25-bedf-5bcd80e281fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:50.964318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1869305-2e58-4b25-bedf-5bcd80e281fc admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:50.968369 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd144d-f2f2-49e3-ae37-bbb6de950e70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:50.969216 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd144d-f2f2-49e3-ae37-bbb6de950e70 service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:50.969502 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd144d-f2f2-49e3-ae37-bbb6de950e70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:50.969780 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd144d-f2f2-49e3-ae37-bbb6de950e70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:50.980296 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcd144d-f2f2-49e3-ae37-bbb6de950e70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:50.981994 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcd144d-f2f2-49e3-ae37-bbb6de950e70 service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:50.992903 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1408/2789] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:51.232858 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68caab2c-5ff1-421b-8276-8cf0a439c936 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:51.233923 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-68caab2c-5ff1-421b-8276-8cf0a439c936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:51.254955 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68caab2c-5ff1-421b-8276-8cf0a439c936 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:51.260499 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d8a830-8547-42b2-a917-7aa806cf17a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:51.261211 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8a830-8547-42b2-a917-7aa806cf17a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:51.261492 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8a830-8547-42b2-a917-7aa806cf17a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:51.261767 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d8a830-8547-42b2-a917-7aa806cf17a9 admin admin] PATH_INFO: `/v3/users/aa92e046c3274a29a8aa8857d7570750` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:51.263308 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d8a830-8547-42b2-a917-7aa806cf17a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa92e046c3274a29a8aa8857d7570750)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:51.265273 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d8a830-8547-42b2-a917-7aa806cf17a9 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:51.330392 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-44d8a830-8547-42b2-a917-7aa806cf17a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa92e046c3274a29a8aa8857d7570750'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:51.331936 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:51.347156 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd144d-f2f2-49e3-ae37-bbb6de950e70 service neutron] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:51.348438 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-9fcd144d-f2f2-49e3-ae37-bbb6de950e70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:51.358643 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-44d8a830-8547-42b2-a917-7aa806cf17a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa92e046c3274a29a8aa8857d7570750'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:51.372911 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-44d8a830-8547-42b2-a917-7aa806cf17a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa92e046c3274a29a8aa8857d7570750'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:51.373316 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd144d-f2f2-49e3-ae37-bbb6de950e70 service neutron] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:51.381550 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-44d8a830-8547-42b2-a917-7aa806cf17a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa92e046c3274a29a8aa8857d7570750'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:51.382278 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-44d8a830-8547-42b2-a917-7aa806cf17a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa92e046c3274a29a8aa8857d7570750'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:51.384756 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a4f67a-4311-4f1d-9e55-0d3c053f437a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:51.386586 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a4f67a-4311-4f1d-9e55-0d3c053f437a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:51.387741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a4f67a-4311-4f1d-9e55-0d3c053f437a admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:51.389619 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a4f67a-4311-4f1d-9e55-0d3c053f437a admin admin] PATH_INFO: `/v3/projects/ec3c28c54bc041c088bbcc71521b0a4d` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:51.390840 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1382/2790] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:31:51 2022] DELETE /identity/v3/users/aa92e046c3274a29a8aa8857d7570750 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:51.393196 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a4f67a-4311-4f1d-9e55-0d3c053f437a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec3c28c54bc041c088bbcc71521b0a4d)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:51.395429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a4f67a-4311-4f1d-9e55-0d3c053f437a admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:51.413234 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:51.455961 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f8a4f67a-4311-4f1d-9e55-0d3c053f437a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec3c28c54bc041c088bbcc71521b0a4d', 'request_id': 'req-f8a4f67a-4311-4f1d-9e55-0d3c053f437a'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:51.463300 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f8a4f67a-4311-4f1d-9e55-0d3c053f437a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec3c28c54bc041c088bbcc71521b0a4d', 'request_id': 'req-f8a4f67a-4311-4f1d-9e55-0d3c053f437a'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:51.476751 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f8a4f67a-4311-4f1d-9e55-0d3c053f437a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec3c28c54bc041c088bbcc71521b0a4d', 'request_id': 'req-f8a4f67a-4311-4f1d-9e55-0d3c053f437a'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:51.491585 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f8a4f67a-4311-4f1d-9e55-0d3c053f437a admin admin] The token cache is being invalidate because project ec3c28c54bc041c088bbcc71521b0a4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:31:51.491871 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f8a4f67a-4311-4f1d-9e55-0d3c053f437a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:51.492529 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f8a4f67a-4311-4f1d-9e55-0d3c053f437a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:51.494168 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1409/2791] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:31:51 2022] DELETE /identity/v3/projects/ec3c28c54bc041c088bbcc71521b0a4d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:51.499358 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af601d93-8656-47b5-9358-3157411fd193 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:51.499645 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af601d93-8656-47b5-9358-3157411fd193 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:51.499917 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af601d93-8656-47b5-9358-3157411fd193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:51.526056 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-af601d93-8656-47b5-9358-3157411fd193 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:51.527275 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-af601d93-8656-47b5-9358-3157411fd193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:51.600028 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a4f67a-4311-4f1d-9e55-0d3c053f437a admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:51.601550 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-f8a4f67a-4311-4f1d-9e55-0d3c053f437a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:51.609030 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1383/2792] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:51.610578 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ee8bb-60da-4b76-ab24-3577205a0928 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:51.610925 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ee8bb-60da-4b76-ab24-3577205a0928 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:51.611212 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ee8bb-60da-4b76-ab24-3577205a0928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:51.620896 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a4f67a-4311-4f1d-9e55-0d3c053f437a admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:51.629577 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399a0960-6633-4bbc-8006-a0fd649308ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:51.629577 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a0960-6633-4bbc-8006-a0fd649308ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:51.630089 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a0960-6633-4bbc-8006-a0fd649308ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:51.630450 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399a0960-6633-4bbc-8006-a0fd649308ac admin admin] PATH_INFO: `/v3/projects/9966ab8ffdcf4e828553261c05fbb5a9` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:51.634447 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399a0960-6633-4bbc-8006-a0fd649308ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=9966ab8ffdcf4e828553261c05fbb5a9)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:51.635644 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399a0960-6633-4bbc-8006-a0fd649308ac admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:51.636424 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-ae2ee8bb-60da-4b76-ab24-3577205a0928 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:51.637874 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-ae2ee8bb-60da-4b76-ab24-3577205a0928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:51.648707 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1384/2793] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:51.652902 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ee8bb-60da-4b76-ab24-3577205a0928 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:51.662320 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ee8bb-60da-4b76-ab24-3577205a0928 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:51.667205 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b707c7-f820-4864-84e5-7cccb19b3a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:51.667937 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b707c7-f820-4864-84e5-7cccb19b3a19 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:51.668404 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b707c7-f820-4864-84e5-7cccb19b3a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:51.668637 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b707c7-f820-4864-84e5-7cccb19b3a19 admin admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:51.669505 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b707c7-f820-4864-84e5-7cccb19b3a19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:31:51.669906 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b707c7-f820-4864-84e5-7cccb19b3a19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:51.671588 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b707c7-f820-4864-84e5-7cccb19b3a19 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:51.689695 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1385/2794] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:31:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:31:51.690746 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b707c7-f820-4864-84e5-7cccb19b3a19 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:51.702097 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b707c7-f820-4864-84e5-7cccb19b3a19 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:51.708429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-399a0960-6633-4bbc-8006-a0fd649308ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9966ab8ffdcf4e828553261c05fbb5a9', 'request_id': 'req-399a0960-6633-4bbc-8006-a0fd649308ac'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:51.716415 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-399a0960-6633-4bbc-8006-a0fd649308ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9966ab8ffdcf4e828553261c05fbb5a9', 'request_id': 'req-399a0960-6633-4bbc-8006-a0fd649308ac'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:51.727903 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-399a0960-6633-4bbc-8006-a0fd649308ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9966ab8ffdcf4e828553261c05fbb5a9', 'request_id': 'req-399a0960-6633-4bbc-8006-a0fd649308ac'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:51.740673 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-399a0960-6633-4bbc-8006-a0fd649308ac admin admin] The token cache is being invalidate because project 9966ab8ffdcf4e828553261c05fbb5a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:31:51.740973 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-399a0960-6633-4bbc-8006-a0fd649308ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:51.741644 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-399a0960-6633-4bbc-8006-a0fd649308ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:51.743484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1410/2795] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:31:51 2022] DELETE /identity/v3/projects/9966ab8ffdcf4e828553261c05fbb5a9 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:31:51.748671 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31566e9-18be-4f4a-9130-6b855ff6fa09 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:51.749058 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31566e9-18be-4f4a-9130-6b855ff6fa09 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:51.749434 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31566e9-18be-4f4a-9130-6b855ff6fa09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:51.771144 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-f31566e9-18be-4f4a-9130-6b855ff6fa09 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:51.772154 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ffb878-f0cc-46b7-abfa-01e10ec98452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:51.773012 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ffb878-f0cc-46b7-abfa-01e10ec98452 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:51.773254 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-f31566e9-18be-4f4a-9130-6b855ff6fa09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:51.773487 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ffb878-f0cc-46b7-abfa-01e10ec98452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:51.773826 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ffb878-f0cc-46b7-abfa-01e10ec98452 admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:51.775653 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ffb878-f0cc-46b7-abfa-01e10ec98452 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:51.777919 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ffb878-f0cc-46b7-abfa-01e10ec98452 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:51.782733 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1411/2796] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:31:51.791331 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594293fb-8259-4412-8886-85a7a568b5ae None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:51.791827 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594293fb-8259-4412-8886-85a7a568b5ae None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:51.791950 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594293fb-8259-4412-8886-85a7a568b5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:51.814716 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-594293fb-8259-4412-8886-85a7a568b5ae None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:51.816038 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-594293fb-8259-4412-8886-85a7a568b5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:51.821171 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1386/2797] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:31:51 2022] POST /identity/v3/projects => generated 391 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:31:51.828792 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ffb878-f0cc-46b7-abfa-01e10ec98452 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:51.828792 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-77ffb878-f0cc-46b7-abfa-01e10ec98452 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:51.874355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:51.882462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1412/2798] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:51.887381 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594293fb-8259-4412-8886-85a7a568b5ae None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:51.897255 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594293fb-8259-4412-8886-85a7a568b5ae None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:51.902218 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc44ecab-38b5-452f-9035-7524ab65114e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:51.903561 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44ecab-38b5-452f-9035-7524ab65114e admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:51.903838 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44ecab-38b5-452f-9035-7524ab65114e admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:51.904105 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc44ecab-38b5-452f-9035-7524ab65114e admin admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:51.904832 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc44ecab-38b5-452f-9035-7524ab65114e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:31:51.905167 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc44ecab-38b5-452f-9035-7524ab65114e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:51.906786 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc44ecab-38b5-452f-9035-7524ab65114e admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:51.909369 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ffb878-f0cc-46b7-abfa-01e10ec98452 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:51.913969 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd83ab-aee9-4125-a247-aaf1daeaf325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:51.914638 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd83ab-aee9-4125-a247-aaf1daeaf325 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:51.914917 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd83ab-aee9-4125-a247-aaf1daeaf325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:51.915189 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd83ab-aee9-4125-a247-aaf1daeaf325 admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:51.915852 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fd83ab-aee9-4125-a247-aaf1daeaf325 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:51.917816 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fd83ab-aee9-4125-a247-aaf1daeaf325 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:51.922740 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1413/2799] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:31:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:31:51.923638 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc44ecab-38b5-452f-9035-7524ab65114e admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:51.933498 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc44ecab-38b5-452f-9035-7524ab65114e admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:51.938673 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbf5488-87bb-4e3a-9b31-58e16ef8fb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:51.939324 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf5488-87bb-4e3a-9b31-58e16ef8fb78 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:51.943314 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1387/2800] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:31:51 2022] POST /identity/v3/users => generated 296 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:51.943473 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf5488-87bb-4e3a-9b31-58e16ef8fb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:51.943473 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf5488-87bb-4e3a-9b31-58e16ef8fb78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:51.943473 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbf5488-87bb-4e3a-9b31-58e16ef8fb78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:51.943473 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbf5488-87bb-4e3a-9b31-58e16ef8fb78 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:51.943881 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd83ab-aee9-4125-a247-aaf1daeaf325 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:51.946425 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:51.954034 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd83ab-aee9-4125-a247-aaf1daeaf325 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:51.959712 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f801bcb-d960-4a59-a802-923fb8a94a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:51.960450 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f801bcb-d960-4a59-a802-923fb8a94a0d admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:51.960748 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f801bcb-d960-4a59-a802-923fb8a94a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:51.961043 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f801bcb-d960-4a59-a802-923fb8a94a0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:51.961946 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f801bcb-d960-4a59-a802-923fb8a94a0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:51.963898 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f801bcb-d960-4a59-a802-923fb8a94a0d admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:51.978393 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1388/2801] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:31:51.978593 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1414/2802] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:31:51 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:51.980814 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbf5488-87bb-4e3a-9b31-58e16ef8fb78 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:51.981094 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f801bcb-d960-4a59-a802-923fb8a94a0d admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:51.991745 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f801bcb-d960-4a59-a802-923fb8a94a0d admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:52.050756 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbf5488-87bb-4e3a-9b31-58e16ef8fb78 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:52.055844 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b1aef6-32fd-4e5d-9fa2-764785f2e7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:52.056528 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b1aef6-32fd-4e5d-9fa2-764785f2e7ed admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:52.056719 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032ed4e-f10c-478b-8ea6-5e1838f4fc31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:52.056840 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b1aef6-32fd-4e5d-9fa2-764785f2e7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:52.057102 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b1aef6-32fd-4e5d-9fa2-764785f2e7ed admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:52.057432 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032ed4e-f10c-478b-8ea6-5e1838f4fc31 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:52.057717 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032ed4e-f10c-478b-8ea6-5e1838f4fc31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:52.057826 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b1aef6-32fd-4e5d-9fa2-764785f2e7ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:52.057993 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032ed4e-f10c-478b-8ea6-5e1838f4fc31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:52.058768 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4032ed4e-f10c-478b-8ea6-5e1838f4fc31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:52.059729 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b1aef6-32fd-4e5d-9fa2-764785f2e7ed admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:52.060610 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4032ed4e-f10c-478b-8ea6-5e1838f4fc31 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:52.076456 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1389/2803] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:51 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:52.081139 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032ed4e-f10c-478b-8ea6-5e1838f4fc31 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:52.085872 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1415/2804] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:31:51 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 03:31:52.087079 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b1aef6-32fd-4e5d-9fa2-764785f2e7ed admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:52.092116 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032ed4e-f10c-478b-8ea6-5e1838f4fc31 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:52.096773 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb74643-415c-45f3-851e-816af714e907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:52.097351 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b1aef6-32fd-4e5d-9fa2-764785f2e7ed admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:52.097724 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb74643-415c-45f3-851e-816af714e907 admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:52.097997 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb74643-415c-45f3-851e-816af714e907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:52.098274 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb74643-415c-45f3-851e-816af714e907 admin admin] PATH_INFO: `/v3/projects/14f492b5b70c4736ab571b5b3d331bf8/users/b8b2deb6f91548d8b831ce892def114c/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:52.102074 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6e0553-0c34-459d-8c34-b8e5a5270b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:52.102747 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e0553-0c34-459d-8c34-b8e5a5270b4d admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:52.103012 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e0553-0c34-459d-8c34-b8e5a5270b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:52.103284 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e0553-0c34-459d-8c34-b8e5a5270b4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:52.104005 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6e0553-0c34-459d-8c34-b8e5a5270b4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:52.105850 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6e0553-0c34-459d-8c34-b8e5a5270b4d admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:52.114174 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb74643-415c-45f3-851e-816af714e907 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14f492b5b70c4736ab571b5b3d331bf8, user_id=b8b2deb6f91548d8b831ce892def114c, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:52.116004 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1416/2805] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:52.117890 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb74643-415c-45f3-851e-816af714e907 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:52.120020 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6e0553-0c34-459d-8c34-b8e5a5270b4d admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:52.130784 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6e0553-0c34-459d-8c34-b8e5a5270b4d admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:52.133062 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1390/2806] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:31:52 2022] PUT /identity/v3/projects/14f492b5b70c4736ab571b5b3d331bf8/users/b8b2deb6f91548d8b831ce892def114c/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:31:52.144780 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5501bdfb-83e9-4479-916b-2bcd2af8477d None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:52.145026 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5501bdfb-83e9-4479-916b-2bcd2af8477d None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:52.145273 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5501bdfb-83e9-4479-916b-2bcd2af8477d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:52.182084 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-5501bdfb-83e9-4479-916b-2bcd2af8477d None None] MFA Rules not processed for user `b8b2deb6f91548d8b831ce892def114c`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:52.183293 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-5501bdfb-83e9-4479-916b-2bcd2af8477d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:52.192977 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8210a7-7be8-4818-90f1-d9f721c9ffe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:52.193784 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8210a7-7be8-4818-90f1-d9f721c9ffe2 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:52.194067 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8210a7-7be8-4818-90f1-d9f721c9ffe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:52.194342 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8210a7-7be8-4818-90f1-d9f721c9ffe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:52.195187 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8210a7-7be8-4818-90f1-d9f721c9ffe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:52.196824 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8210a7-7be8-4818-90f1-d9f721c9ffe2 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:52.210522 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1417/2807] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:52 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 03:31:52.211539 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8210a7-7be8-4818-90f1-d9f721c9ffe2 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:52.221998 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8210a7-7be8-4818-90f1-d9f721c9ffe2 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:52.226653 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9598b7f7-8e9f-47ee-9148-31dffe670a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:52.227530 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598b7f7-8e9f-47ee-9148-31dffe670a34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:52.227887 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598b7f7-8e9f-47ee-9148-31dffe670a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:52.228082 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9598b7f7-8e9f-47ee-9148-31dffe670a34 admin admin] PATH_INFO: `/v3/projects/5dd967b52989488b91e535ac5f8a93d9/users/0d051848c7014e6495111b4f13afc000/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:52.243884 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9598b7f7-8e9f-47ee-9148-31dffe670a34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dd967b52989488b91e535ac5f8a93d9, user_id=0d051848c7014e6495111b4f13afc000, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:52.247113 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9598b7f7-8e9f-47ee-9148-31dffe670a34 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:52.253505 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1391/2808] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:52.266190 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5501bdfb-83e9-4479-916b-2bcd2af8477d None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:52.267513 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1418/2809] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:31:52 2022] PUT /identity/v3/projects/5dd967b52989488b91e535ac5f8a93d9/users/0d051848c7014e6495111b4f13afc000/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:31:52.267640 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-5501bdfb-83e9-4479-916b-2bcd2af8477d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:52.277959 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6b201-1c06-41b7-a951-f88b251b0e14 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:52.278351 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6b201-1c06-41b7-a951-f88b251b0e14 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:52.278871 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6b201-1c06-41b7-a951-f88b251b0e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:52.282829 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:52.311448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-d1a6b201-1c06-41b7-a951-f88b251b0e14 None None] MFA Rules not processed for user `0d051848c7014e6495111b4f13afc000`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:52.312615 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-d1a6b201-1c06-41b7-a951-f88b251b0e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:52.328097 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5501bdfb-83e9-4479-916b-2bcd2af8477d None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:52.338204 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:52.383787 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585cd786-1e3d-4847-a9cc-baabc5bcecb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:52.384527 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cd786-1e3d-4847-a9cc-baabc5bcecb3 service cinder] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:52.384936 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cd786-1e3d-4847-a9cc-baabc5bcecb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:52.385236 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cd786-1e3d-4847-a9cc-baabc5bcecb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:52.387646 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1419/2810] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:52.397589 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a6b201-1c06-41b7-a951-f88b251b0e14 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:52.399036 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-d1a6b201-1c06-41b7-a951-f88b251b0e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:52.447538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585cd786-1e3d-4847-a9cc-baabc5bcecb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:52.449178 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585cd786-1e3d-4847-a9cc-baabc5bcecb3 service cinder] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:52.458438 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1392/2811] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:52 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:31:52.459292 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585cd786-1e3d-4847-a9cc-baabc5bcecb3 service cinder] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:52.460589 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-585cd786-1e3d-4847-a9cc-baabc5bcecb3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:52.469605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a6b201-1c06-41b7-a951-f88b251b0e14 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:52.473816 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585cd786-1e3d-4847-a9cc-baabc5bcecb3 service cinder] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:52.474854 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656458d6-44e6-49d2-8fc5-a7a0e5779cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:52.475846 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656458d6-44e6-49d2-8fc5-a7a0e5779cd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:52.476126 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656458d6-44e6-49d2-8fc5-a7a0e5779cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:52.476411 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656458d6-44e6-49d2-8fc5-a7a0e5779cd2 admin admin] PATH_INFO: `/v3/users/ae63e5c5006640fdbfdbb90ce6f404f7` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:52.477589 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16bd98b-6ff9-40f9-ab2b-3830aa1eec5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:52.477805 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656458d6-44e6-49d2-8fc5-a7a0e5779cd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae63e5c5006640fdbfdbb90ce6f404f7)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:52.478238 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16bd98b-6ff9-40f9-ab2b-3830aa1eec5a service cinder] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:52.478656 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16bd98b-6ff9-40f9-ab2b-3830aa1eec5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:52.478935 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16bd98b-6ff9-40f9-ab2b-3830aa1eec5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:52.479910 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656458d6-44e6-49d2-8fc5-a7a0e5779cd2 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:52.489060 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16bd98b-6ff9-40f9-ab2b-3830aa1eec5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:52.490494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16bd98b-6ff9-40f9-ab2b-3830aa1eec5a service cinder] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:52.500772 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1393/2812] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:52.536461 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-656458d6-44e6-49d2-8fc5-a7a0e5779cd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae63e5c5006640fdbfdbb90ce6f404f7'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:52.537281 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-656458d6-44e6-49d2-8fc5-a7a0e5779cd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae63e5c5006640fdbfdbb90ce6f404f7'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:52.549028 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-656458d6-44e6-49d2-8fc5-a7a0e5779cd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae63e5c5006640fdbfdbb90ce6f404f7'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:52.549888 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-656458d6-44e6-49d2-8fc5-a7a0e5779cd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae63e5c5006640fdbfdbb90ce6f404f7'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:52.564533 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-656458d6-44e6-49d2-8fc5-a7a0e5779cd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae63e5c5006640fdbfdbb90ce6f404f7'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:52.579144 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1420/2813] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:31:52 2022] DELETE /identity/v3/users/ae63e5c5006640fdbfdbb90ce6f404f7 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:31:52.583663 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e934d-d84a-443e-945a-81d900b7d159 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:52.583952 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e934d-d84a-443e-945a-81d900b7d159 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:52.584227 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e934d-d84a-443e-945a-81d900b7d159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:52.603158 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-493e934d-d84a-443e-945a-81d900b7d159 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:52.604386 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-493e934d-d84a-443e-945a-81d900b7d159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:52.679980 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1394/2814] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:52.693163 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656458d6-44e6-49d2-8fc5-a7a0e5779cd2 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:52.695128 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-656458d6-44e6-49d2-8fc5-a7a0e5779cd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:52.760235 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:52.774302 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656458d6-44e6-49d2-8fc5-a7a0e5779cd2 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:52.792926 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ef2a62-2f21-4738-967c-1344b5ea6acf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:52.793859 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef2a62-2f21-4738-967c-1344b5ea6acf service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:52.794148 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef2a62-2f21-4738-967c-1344b5ea6acf service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:52.794878 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ef2a62-2f21-4738-967c-1344b5ea6acf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:52.809935 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ef2a62-2f21-4738-967c-1344b5ea6acf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:52.811529 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ef2a62-2f21-4738-967c-1344b5ea6acf service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:52.824576 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1421/2815] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:52.928753 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e934d-d84a-443e-945a-81d900b7d159 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:52.930347 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-493e934d-d84a-443e-945a-81d900b7d159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:52.943046 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e934d-d84a-443e-945a-81d900b7d159 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:52.948188 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0fab1-efd7-4d51-860e-d5381753f7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:52.949190 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0fab1-efd7-4d51-860e-d5381753f7a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:52.949581 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0fab1-efd7-4d51-860e-d5381753f7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:52.949917 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be0fab1-efd7-4d51-860e-d5381753f7a5 admin admin] PATH_INFO: `/v3/users/ecc14b3653d54d3390144d876feed9ad` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:52.951243 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be0fab1-efd7-4d51-860e-d5381753f7a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecc14b3653d54d3390144d876feed9ad)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:52.953807 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be0fab1-efd7-4d51-860e-d5381753f7a5 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:52.956209 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=100 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:53.017122 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-0be0fab1-efd7-4d51-860e-d5381753f7a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecc14b3653d54d3390144d876feed9ad'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:53.030771 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-0be0fab1-efd7-4d51-860e-d5381753f7a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecc14b3653d54d3390144d876feed9ad'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:53.040313 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-0be0fab1-efd7-4d51-860e-d5381753f7a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecc14b3653d54d3390144d876feed9ad'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:53.047680 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-0be0fab1-efd7-4d51-860e-d5381753f7a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecc14b3653d54d3390144d876feed9ad'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:53.048459 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-0be0fab1-efd7-4d51-860e-d5381753f7a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecc14b3653d54d3390144d876feed9ad'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:53.055026 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1395/2816] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:31:52 2022] DELETE /identity/v3/users/ecc14b3653d54d3390144d876feed9ad => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:53.059829 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3614459-883d-4d67-8412-feaf4fabc35e None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:53.060161 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3614459-883d-4d67-8412-feaf4fabc35e None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:53.060428 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3614459-883d-4d67-8412-feaf4fabc35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:53.078846 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-b3614459-883d-4d67-8412-feaf4fabc35e None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:53.080070 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-b3614459-883d-4d67-8412-feaf4fabc35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:53.136604 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0fab1-efd7-4d51-860e-d5381753f7a5 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:53.138734 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-0be0fab1-efd7-4d51-860e-d5381753f7a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:53.152674 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1422/2817] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:53.163074 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3614459-883d-4d67-8412-feaf4fabc35e None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:53.166698 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-b3614459-883d-4d67-8412-feaf4fabc35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:53.212491 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be0fab1-efd7-4d51-860e-d5381753f7a5 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:53.217363 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395d867-49ca-4f16-bd58-4940915e4f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:53.218239 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395d867-49ca-4f16-bd58-4940915e4f08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:53.218516 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395d867-49ca-4f16-bd58-4940915e4f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:53.218787 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2395d867-49ca-4f16-bd58-4940915e4f08 admin admin] PATH_INFO: `/v3/projects/868f8959de474408b53b31d7cbc49325` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:53.220091 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2395d867-49ca-4f16-bd58-4940915e4f08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=868f8959de474408b53b31d7cbc49325)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:53.222125 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2395d867-49ca-4f16-bd58-4940915e4f08 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:53.229658 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3614459-883d-4d67-8412-feaf4fabc35e None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:53.233316 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa397d5-d27c-439b-9cd8-ebeb2c123128 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:53.233982 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa397d5-d27c-439b-9cd8-ebeb2c123128 service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:53.234229 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa397d5-d27c-439b-9cd8-ebeb2c123128 service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:53.234497 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa397d5-d27c-439b-9cd8-ebeb2c123128 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:53.243908 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa397d5-d27c-439b-9cd8-ebeb2c123128 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:53.245394 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa397d5-d27c-439b-9cd8-ebeb2c123128 service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:53.255468 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1423/2818] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:31:53.276913 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-2395d867-49ca-4f16-bd58-4940915e4f08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '868f8959de474408b53b31d7cbc49325', 'request_id': 'req-2395d867-49ca-4f16-bd58-4940915e4f08'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:53.284349 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-2395d867-49ca-4f16-bd58-4940915e4f08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '868f8959de474408b53b31d7cbc49325', 'request_id': 'req-2395d867-49ca-4f16-bd58-4940915e4f08'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:53.290982 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-2395d867-49ca-4f16-bd58-4940915e4f08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '868f8959de474408b53b31d7cbc49325', 'request_id': 'req-2395d867-49ca-4f16-bd58-4940915e4f08'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:53.313402 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-2395d867-49ca-4f16-bd58-4940915e4f08 admin admin] The token cache is being invalidate because project 868f8959de474408b53b31d7cbc49325 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:31:53.313684 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-2395d867-49ca-4f16-bd58-4940915e4f08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:53.314313 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-2395d867-49ca-4f16-bd58-4940915e4f08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:53.316045 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1396/2819] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:31:53 2022] DELETE /identity/v3/projects/868f8959de474408b53b31d7cbc49325 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:53.320687 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa397d5-d27c-439b-9cd8-ebeb2c123128 service neutron] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:53.331511 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa397d5-d27c-439b-9cd8-ebeb2c123128 service neutron] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:53.399949 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4631d112-5c2f-4104-a5fe-f7b1aa2904dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:53.399949 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4631d112-5c2f-4104-a5fe-f7b1aa2904dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:53.399949 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4631d112-5c2f-4104-a5fe-f7b1aa2904dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:53.399949 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4631d112-5c2f-4104-a5fe-f7b1aa2904dd admin admin] PATH_INFO: `/v3/users/8a753b1bfac849be9e647b378462e917` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:53.401496 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4631d112-5c2f-4104-a5fe-f7b1aa2904dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a753b1bfac849be9e647b378462e917)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:53.403081 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4631d112-5c2f-4104-a5fe-f7b1aa2904dd admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:53.444226 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:53.458636 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-4631d112-5c2f-4104-a5fe-f7b1aa2904dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a753b1bfac849be9e647b378462e917'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:53.459524 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-4631d112-5c2f-4104-a5fe-f7b1aa2904dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a753b1bfac849be9e647b378462e917'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:53.467791 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-4631d112-5c2f-4104-a5fe-f7b1aa2904dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a753b1bfac849be9e647b378462e917'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:53.468493 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-4631d112-5c2f-4104-a5fe-f7b1aa2904dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a753b1bfac849be9e647b378462e917'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:53.481416 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-4631d112-5c2f-4104-a5fe-f7b1aa2904dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a753b1bfac849be9e647b378462e917'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:53.493745 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1424/2820] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:31:53 2022] DELETE /identity/v3/users/8a753b1bfac849be9e647b378462e917 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:31:53.549055 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395d867-49ca-4f16-bd58-4940915e4f08 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:53.550789 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-2395d867-49ca-4f16-bd58-4940915e4f08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:53.567487 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2395d867-49ca-4f16-bd58-4940915e4f08 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:53.579625 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:53.636385 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05866831-12e9-4613-848b-98202079ec1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:53.637479 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05866831-12e9-4613-848b-98202079ec1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:53.637996 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05866831-12e9-4613-848b-98202079ec1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:53.638330 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05866831-12e9-4613-848b-98202079ec1e admin admin] PATH_INFO: `/v3/projects/cfb0f434e1b84a31890d33378c2fc8ee` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:53.639548 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05866831-12e9-4613-848b-98202079ec1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfb0f434e1b84a31890d33378c2fc8ee)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:53.641550 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05866831-12e9-4613-848b-98202079ec1e admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:53.691342 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-05866831-12e9-4613-848b-98202079ec1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfb0f434e1b84a31890d33378c2fc8ee', 'request_id': 'req-05866831-12e9-4613-848b-98202079ec1e'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:53.697666 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-05866831-12e9-4613-848b-98202079ec1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfb0f434e1b84a31890d33378c2fc8ee', 'request_id': 'req-05866831-12e9-4613-848b-98202079ec1e'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:53.703789 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-05866831-12e9-4613-848b-98202079ec1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfb0f434e1b84a31890d33378c2fc8ee', 'request_id': 'req-05866831-12e9-4613-848b-98202079ec1e'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:53.722832 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-05866831-12e9-4613-848b-98202079ec1e admin admin] The token cache is being invalidate because project cfb0f434e1b84a31890d33378c2fc8ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:31:53.723106 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-05866831-12e9-4613-848b-98202079ec1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:53.723791 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-05866831-12e9-4613-848b-98202079ec1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:53.725671 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1397/2821] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:31:53 2022] DELETE /identity/v3/projects/cfb0f434e1b84a31890d33378c2fc8ee => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:53.729585 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4631d112-5c2f-4104-a5fe-f7b1aa2904dd admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:53.731130 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-4631d112-5c2f-4104-a5fe-f7b1aa2904dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:53.744081 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4631d112-5c2f-4104-a5fe-f7b1aa2904dd admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:53.787768 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05866831-12e9-4613-848b-98202079ec1e admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:53.789872 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-05866831-12e9-4613-848b-98202079ec1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:53.815359 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e5b361-1230-47ee-a3c2-691bfc8dd067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:53.816296 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5b361-1230-47ee-a3c2-691bfc8dd067 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:53.816579 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5b361-1230-47ee-a3c2-691bfc8dd067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:53.816854 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5b361-1230-47ee-a3c2-691bfc8dd067 admin admin] PATH_INFO: `/v3/users/ac666eb623f34c989593ccb0745000f2` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:53.818166 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e5b361-1230-47ee-a3c2-691bfc8dd067 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac666eb623f34c989593ccb0745000f2)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:53.820498 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e5b361-1230-47ee-a3c2-691bfc8dd067 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:53.885000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05866831-12e9-4613-848b-98202079ec1e admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:53.890818 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408b6236-79e0-48a4-9a45-e664ab876ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:53.892385 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b6236-79e0-48a4-9a45-e664ab876ed9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:53.892715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b6236-79e0-48a4-9a45-e664ab876ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:53.892913 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b6236-79e0-48a4-9a45-e664ab876ed9 admin admin] PATH_INFO: `/v3/projects/f986b88547aa40eb8f0fc5ba2594afea` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:53.894562 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408b6236-79e0-48a4-9a45-e664ab876ed9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f986b88547aa40eb8f0fc5ba2594afea)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:53.896480 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408b6236-79e0-48a4-9a45-e664ab876ed9 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:53.900088 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-02e5b361-1230-47ee-a3c2-691bfc8dd067 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac666eb623f34c989593ccb0745000f2'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:53.900907 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-02e5b361-1230-47ee-a3c2-691bfc8dd067 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac666eb623f34c989593ccb0745000f2'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:53.912285 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-02e5b361-1230-47ee-a3c2-691bfc8dd067 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac666eb623f34c989593ccb0745000f2'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:53.913006 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-02e5b361-1230-47ee-a3c2-691bfc8dd067 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac666eb623f34c989593ccb0745000f2'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:53.926924 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-02e5b361-1230-47ee-a3c2-691bfc8dd067 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac666eb623f34c989593ccb0745000f2'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:53.948266 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1425/2822] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:31:53 2022] DELETE /identity/v3/users/ac666eb623f34c989593ccb0745000f2 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:53.969955 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:53.977414 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-408b6236-79e0-48a4-9a45-e664ab876ed9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f986b88547aa40eb8f0fc5ba2594afea', 'request_id': 'req-408b6236-79e0-48a4-9a45-e664ab876ed9'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:53.986506 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-408b6236-79e0-48a4-9a45-e664ab876ed9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f986b88547aa40eb8f0fc5ba2594afea', 'request_id': 'req-408b6236-79e0-48a4-9a45-e664ab876ed9'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:53.996591 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-408b6236-79e0-48a4-9a45-e664ab876ed9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f986b88547aa40eb8f0fc5ba2594afea', 'request_id': 'req-408b6236-79e0-48a4-9a45-e664ab876ed9'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:54.019285 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-408b6236-79e0-48a4-9a45-e664ab876ed9 admin admin] The token cache is being invalidate because project f986b88547aa40eb8f0fc5ba2594afea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:31:54.019579 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-408b6236-79e0-48a4-9a45-e664ab876ed9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:54.020618 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-408b6236-79e0-48a4-9a45-e664ab876ed9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:54.025591 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1398/2823] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:31:53 2022] DELETE /identity/v3/projects/f986b88547aa40eb8f0fc5ba2594afea => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:31:54.032032 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b648297-4ded-49e2-94f1-e19d24470860 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:54.034525 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b648297-4ded-49e2-94f1-e19d24470860 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:54.035228 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b648297-4ded-49e2-94f1-e19d24470860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:54.081263 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-2b648297-4ded-49e2-94f1-e19d24470860 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:54.081263 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-2b648297-4ded-49e2-94f1-e19d24470860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:54.127346 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:54.186564 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1426/2824] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:54.195022 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c559fad-01f3-431f-a708-9c507fdd6905 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:54.195589 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c559fad-01f3-431f-a708-9c507fdd6905 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:54.195894 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c559fad-01f3-431f-a708-9c507fdd6905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:54.228260 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-0c559fad-01f3-431f-a708-9c507fdd6905 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:54.229785 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-0c559fad-01f3-431f-a708-9c507fdd6905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:54.249442 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1399/2825] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:54.251556 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b648297-4ded-49e2-94f1-e19d24470860 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:54.267626 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b648297-4ded-49e2-94f1-e19d24470860 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:54.276903 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f82f26-e3aa-4601-8773-3e725e731543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:54.277642 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f82f26-e3aa-4601-8773-3e725e731543 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:54.277920 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f82f26-e3aa-4601-8773-3e725e731543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:54.278253 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f82f26-e3aa-4601-8773-3e725e731543 admin admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:54.278919 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f82f26-e3aa-4601-8773-3e725e731543 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:31:54.279223 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f82f26-e3aa-4601-8773-3e725e731543 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:54.281254 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f82f26-e3aa-4601-8773-3e725e731543 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:54.296370 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1427/2826] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:31:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:31:54.302145 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c559fad-01f3-431f-a708-9c507fdd6905 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:54.313037 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c559fad-01f3-431f-a708-9c507fdd6905 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:54.318413 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62654295-5297-4ff2-a166-2f684b8c3d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:54.319086 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62654295-5297-4ff2-a166-2f684b8c3d9b admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:54.319416 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62654295-5297-4ff2-a166-2f684b8c3d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:54.319813 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62654295-5297-4ff2-a166-2f684b8c3d9b admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:54.320542 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62654295-5297-4ff2-a166-2f684b8c3d9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:54.322954 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62654295-5297-4ff2-a166-2f684b8c3d9b admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:54.354295 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f82f26-e3aa-4601-8773-3e725e731543 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:54.356418 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-a1f82f26-e3aa-4601-8773-3e725e731543 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:54.362046 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1400/2827] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:31:54 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:54.364535 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62654295-5297-4ff2-a166-2f684b8c3d9b admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:54.392654 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f82f26-e3aa-4601-8773-3e725e731543 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:54.472914 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62654295-5297-4ff2-a166-2f684b8c3d9b admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:54.478833 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15fc35d-8175-4dec-aff8-cc065ebe6178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:54.479355 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e38353-9e3e-4e81-956c-60e2588d2966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:54.479790 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e38353-9e3e-4e81-956c-60e2588d2966 admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:54.479930 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15fc35d-8175-4dec-aff8-cc065ebe6178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:54.480140 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e38353-9e3e-4e81-956c-60e2588d2966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:54.480264 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15fc35d-8175-4dec-aff8-cc065ebe6178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:54.480357 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e38353-9e3e-4e81-956c-60e2588d2966 admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:54.480594 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15fc35d-8175-4dec-aff8-cc065ebe6178 admin admin] PATH_INFO: `/v3/projects/0585485ff5cc4758a44de84bc3974afd` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:54.481208 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e38353-9e3e-4e81-956c-60e2588d2966 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:54.482024 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15fc35d-8175-4dec-aff8-cc065ebe6178 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0585485ff5cc4758a44de84bc3974afd)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:54.483277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e38353-9e3e-4e81-956c-60e2588d2966 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:54.483941 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15fc35d-8175-4dec-aff8-cc065ebe6178 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:54.505937 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1401/2828] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:31:54 2022] POST /identity/v3/users => generated 286 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:54.510336 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e38353-9e3e-4e81-956c-60e2588d2966 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:54.520856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e38353-9e3e-4e81-956c-60e2588d2966 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:54.525828 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a82e1-f221-4d94-b451-d14dc0a4d6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:54.526512 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a82e1-f221-4d94-b451-d14dc0a4d6e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:54.526776 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a82e1-f221-4d94-b451-d14dc0a4d6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:54.527050 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a82e1-f221-4d94-b451-d14dc0a4d6e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:54.527828 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839a82e1-f221-4d94-b451-d14dc0a4d6e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:54.529483 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839a82e1-f221-4d94-b451-d14dc0a4d6e6 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:54.541853 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1402/2829] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:54.546619 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a82e1-f221-4d94-b451-d14dc0a4d6e6 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:54.551158 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-d15fc35d-8175-4dec-aff8-cc065ebe6178 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0585485ff5cc4758a44de84bc3974afd', 'request_id': 'req-d15fc35d-8175-4dec-aff8-cc065ebe6178'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:54.556613 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a82e1-f221-4d94-b451-d14dc0a4d6e6 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:54.557839 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-d15fc35d-8175-4dec-aff8-cc065ebe6178 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0585485ff5cc4758a44de84bc3974afd', 'request_id': 'req-d15fc35d-8175-4dec-aff8-cc065ebe6178'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:54.567040 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-d15fc35d-8175-4dec-aff8-cc065ebe6178 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0585485ff5cc4758a44de84bc3974afd', 'request_id': 'req-d15fc35d-8175-4dec-aff8-cc065ebe6178'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:54.578521 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-d15fc35d-8175-4dec-aff8-cc065ebe6178 admin admin] The token cache is being invalidate because project 0585485ff5cc4758a44de84bc3974afd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:31:54.578921 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-d15fc35d-8175-4dec-aff8-cc065ebe6178 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:54.579574 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-d15fc35d-8175-4dec-aff8-cc065ebe6178 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:54.581473 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1428/2830] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:31:54 2022] DELETE /identity/v3/projects/0585485ff5cc4758a44de84bc3974afd => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:54.585064 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15fc35d-8175-4dec-aff8-cc065ebe6178 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:54.586578 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-d15fc35d-8175-4dec-aff8-cc065ebe6178 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:54.616875 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d1bfb-2987-4a71-9330-593b630a6a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:54.617488 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d1bfb-2987-4a71-9330-593b630a6a83 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:54.617769 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d1bfb-2987-4a71-9330-593b630a6a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:54.618025 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d1bfb-2987-4a71-9330-593b630a6a83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:54.618766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194d1bfb-2987-4a71-9330-593b630a6a83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:54.620336 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194d1bfb-2987-4a71-9330-593b630a6a83 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:54.635351 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1403/2831] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:54 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:54.636222 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:54.636987 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d1bfb-2987-4a71-9330-593b630a6a83 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:54.638960 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-194d1bfb-2987-4a71-9330-593b630a6a83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:54.652076 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d1bfb-2987-4a71-9330-593b630a6a83 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:54.654350 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15fc35d-8175-4dec-aff8-cc065ebe6178 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:54.657632 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce4d5f1-a653-42c8-ad92-b2a5638d5f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:54.659083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4d5f1-a653-42c8-ad92-b2a5638d5f37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:54.659083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4d5f1-a653-42c8-ad92-b2a5638d5f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:54.659083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce4d5f1-a653-42c8-ad92-b2a5638d5f37 admin admin] PATH_INFO: `/v3/projects/2bab42aa679b4182887da08409099827/users/c338d0e7dce9404597cd0061e408091d/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:54.659380 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d328f6b-e086-45bc-9a20-ea851134bc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:54.660314 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d328f6b-e086-45bc-9a20-ea851134bc9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:54.660596 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d328f6b-e086-45bc-9a20-ea851134bc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:54.660870 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d328f6b-e086-45bc-9a20-ea851134bc9b admin admin] PATH_INFO: `/v3/users/7fb3ab8dc0cb409a99e18cf5c61713be` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:54.662098 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d328f6b-e086-45bc-9a20-ea851134bc9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fb3ab8dc0cb409a99e18cf5c61713be)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:54.664048 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d328f6b-e086-45bc-9a20-ea851134bc9b admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:54.675381 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce4d5f1-a653-42c8-ad92-b2a5638d5f37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bab42aa679b4182887da08409099827, user_id=c338d0e7dce9404597cd0061e408091d, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:54.678139 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce4d5f1-a653-42c8-ad92-b2a5638d5f37 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:54.680499 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:54.696894 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1404/2832] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:31:54 2022] PUT /identity/v3/projects/2bab42aa679b4182887da08409099827/users/c338d0e7dce9404597cd0061e408091d/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:31:54.703168 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd40a4e-99db-4c14-82bf-bd6a1c04695d None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:54.703528 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd40a4e-99db-4c14-82bf-bd6a1c04695d None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:54.703828 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd40a4e-99db-4c14-82bf-bd6a1c04695d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:54.712792 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-2d328f6b-e086-45bc-9a20-ea851134bc9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fb3ab8dc0cb409a99e18cf5c61713be'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:54.713483 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-2d328f6b-e086-45bc-9a20-ea851134bc9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fb3ab8dc0cb409a99e18cf5c61713be'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:54.721643 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-8fd40a4e-99db-4c14-82bf-bd6a1c04695d None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:54.722745 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-2d328f6b-e086-45bc-9a20-ea851134bc9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fb3ab8dc0cb409a99e18cf5c61713be'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:54.723013 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-8fd40a4e-99db-4c14-82bf-bd6a1c04695d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:54.723404 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-2d328f6b-e086-45bc-9a20-ea851134bc9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fb3ab8dc0cb409a99e18cf5c61713be'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:54.736429 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-2d328f6b-e086-45bc-9a20-ea851134bc9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fb3ab8dc0cb409a99e18cf5c61713be'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:54.749652 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1429/2833] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:31:54 2022] DELETE /identity/v3/users/7fb3ab8dc0cb409a99e18cf5c61713be => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:54.785976 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1405/2834] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 03:31:54.796323 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d328f6b-e086-45bc-9a20-ea851134bc9b admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:54.797832 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-2d328f6b-e086-45bc-9a20-ea851134bc9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:54.861622 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d328f6b-e086-45bc-9a20-ea851134bc9b admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:54.865440 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0eb16-4053-4f3c-b549-f57228640d6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:54.866169 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0eb16-4053-4f3c-b549-f57228640d6e service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:54.866521 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0eb16-4053-4f3c-b549-f57228640d6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:54.866869 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0eb16-4053-4f3c-b549-f57228640d6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:54.937504 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a0eb16-4053-4f3c-b549-f57228640d6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:54.939370 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a0eb16-4053-4f3c-b549-f57228640d6e service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:54.949668 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1430/2835] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:55.089997 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd40a4e-99db-4c14-82bf-bd6a1c04695d None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:55.091866 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-8fd40a4e-99db-4c14-82bf-bd6a1c04695d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:55.105893 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd40a4e-99db-4c14-82bf-bd6a1c04695d None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:55.111856 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f192c-a14d-4ba0-8e45-82d5192d5c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:55.112859 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f192c-a14d-4ba0-8e45-82d5192d5c61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:55.113707 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f192c-a14d-4ba0-8e45-82d5192d5c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:55.113965 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f192c-a14d-4ba0-8e45-82d5192d5c61 admin admin] PATH_INFO: `/v3/projects/bc29662523c04e94b2c3e03b34b33f57` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:55.115264 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3f192c-a14d-4ba0-8e45-82d5192d5c61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc29662523c04e94b2c3e03b34b33f57)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:55.117658 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3f192c-a14d-4ba0-8e45-82d5192d5c61 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:55.174296 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-7b3f192c-a14d-4ba0-8e45-82d5192d5c61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc29662523c04e94b2c3e03b34b33f57', 'request_id': 'req-7b3f192c-a14d-4ba0-8e45-82d5192d5c61'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:55.180611 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-7b3f192c-a14d-4ba0-8e45-82d5192d5c61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc29662523c04e94b2c3e03b34b33f57', 'request_id': 'req-7b3f192c-a14d-4ba0-8e45-82d5192d5c61'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:55.187083 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-7b3f192c-a14d-4ba0-8e45-82d5192d5c61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc29662523c04e94b2c3e03b34b33f57', 'request_id': 'req-7b3f192c-a14d-4ba0-8e45-82d5192d5c61'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:55.202248 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-7b3f192c-a14d-4ba0-8e45-82d5192d5c61 admin admin] The token cache is being invalidate because project bc29662523c04e94b2c3e03b34b33f57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:31:55.202537 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-7b3f192c-a14d-4ba0-8e45-82d5192d5c61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:55.205415 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-7b3f192c-a14d-4ba0-8e45-82d5192d5c61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:31:55.207013 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1406/2836] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:31:55 2022] DELETE /identity/v3/projects/bc29662523c04e94b2c3e03b34b33f57 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:55.212972 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39368f8-a0fc-4156-a3bf-d622efd8aa04 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:55.213268 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39368f8-a0fc-4156-a3bf-d622efd8aa04 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:55.213552 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39368f8-a0fc-4156-a3bf-d622efd8aa04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:55.234499 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-b39368f8-a0fc-4156-a3bf-d622efd8aa04 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:55.235690 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-b39368f8-a0fc-4156-a3bf-d622efd8aa04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:55.245561 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:55.308071 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1431/2837] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:55.316165 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d92126-0f14-4e63-811c-bb7aabe3dce8 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:55.316674 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d92126-0f14-4e63-811c-bb7aabe3dce8 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:55.316970 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d92126-0f14-4e63-811c-bb7aabe3dce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:55.349365 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-d9d92126-0f14-4e63-811c-bb7aabe3dce8 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:55.350595 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-d9d92126-0f14-4e63-811c-bb7aabe3dce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:55.361870 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1407/2838] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:55.366021 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39368f8-a0fc-4156-a3bf-d622efd8aa04 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:55.375946 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39368f8-a0fc-4156-a3bf-d622efd8aa04 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:55.380551 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2169c6-4770-47ec-b606-885f0c28580c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:55.381390 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2169c6-4770-47ec-b606-885f0c28580c admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:55.381671 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2169c6-4770-47ec-b606-885f0c28580c admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:55.382037 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2169c6-4770-47ec-b606-885f0c28580c admin admin] PATH_INFO: `/v3/domains` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:55.382909 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2169c6-4770-47ec-b606-885f0c28580c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80409) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:31:55.383218 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2169c6-4770-47ec-b606-885f0c28580c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:55.385263 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2169c6-4770-47ec-b606-885f0c28580c admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:55.397937 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1432/2839] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:31:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 03:31:55.402218 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d92126-0f14-4e63-811c-bb7aabe3dce8 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:55.414466 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d92126-0f14-4e63-811c-bb7aabe3dce8 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:55.419608 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db35a03-65fe-4054-a03d-d0a9735dc70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:55.420271 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db35a03-65fe-4054-a03d-d0a9735dc70f admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:55.420551 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db35a03-65fe-4054-a03d-d0a9735dc70f admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:55.420822 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db35a03-65fe-4054-a03d-d0a9735dc70f admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:55.421566 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db35a03-65fe-4054-a03d-d0a9735dc70f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:55.423548 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db35a03-65fe-4054-a03d-d0a9735dc70f admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:55.451882 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1408/2840] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:31:55 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:55.456360 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2169c6-4770-47ec-b606-885f0c28580c admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:55.467931 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2169c6-4770-47ec-b606-885f0c28580c admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:55.530884 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba904ac7-eeb2-4a44-b59b-98f41ca47d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:55.531600 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba904ac7-eeb2-4a44-b59b-98f41ca47d95 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:55.531874 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba904ac7-eeb2-4a44-b59b-98f41ca47d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:55.532190 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba904ac7-eeb2-4a44-b59b-98f41ca47d95 admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:55.532850 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba904ac7-eeb2-4a44-b59b-98f41ca47d95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:55.534975 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba904ac7-eeb2-4a44-b59b-98f41ca47d95 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:55.559193 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1433/2841] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:31:55 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:55.560650 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db35a03-65fe-4054-a03d-d0a9735dc70f admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:55.571368 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db35a03-65fe-4054-a03d-d0a9735dc70f admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:55.576888 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb86861-2e94-48b5-9657-95fed6936b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:55.577766 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb86861-2e94-48b5-9657-95fed6936b51 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:55.578085 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb86861-2e94-48b5-9657-95fed6936b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:55.578316 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb86861-2e94-48b5-9657-95fed6936b51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:55.579043 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb86861-2e94-48b5-9657-95fed6936b51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:55.580641 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb86861-2e94-48b5-9657-95fed6936b51 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:55.593433 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1409/2842] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:55.598668 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba904ac7-eeb2-4a44-b59b-98f41ca47d95 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:55.611330 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba904ac7-eeb2-4a44-b59b-98f41ca47d95 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:55.617883 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99667fed-7b9e-4d80-95dc-f616e3b9cc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:55.618664 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99667fed-7b9e-4d80-95dc-f616e3b9cc04 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:55.619050 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99667fed-7b9e-4d80-95dc-f616e3b9cc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:55.619465 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99667fed-7b9e-4d80-95dc-f616e3b9cc04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:55.620533 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99667fed-7b9e-4d80-95dc-f616e3b9cc04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:55.622961 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99667fed-7b9e-4d80-95dc-f616e3b9cc04 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:55.635515 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1434/2843] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:55.640113 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb86861-2e94-48b5-9657-95fed6936b51 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:55.650820 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb86861-2e94-48b5-9657-95fed6936b51 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:55.655706 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf5fff4-2388-480a-ba75-3dd9b3ea82e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:55.656685 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5fff4-2388-480a-ba75-3dd9b3ea82e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:55.657314 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5fff4-2388-480a-ba75-3dd9b3ea82e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:55.657314 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5fff4-2388-480a-ba75-3dd9b3ea82e9 admin admin] PATH_INFO: `/v3/projects/1cd50a0a32ae481b82c64c706ebd1bda/users/9e74299d97bc48c59bed66ebea5d64e3/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:55.677179 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf5fff4-2388-480a-ba75-3dd9b3ea82e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cd50a0a32ae481b82c64c706ebd1bda, user_id=9e74299d97bc48c59bed66ebea5d64e3, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:55.679958 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf5fff4-2388-480a-ba75-3dd9b3ea82e9 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:55.697879 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1410/2844] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:31:55 2022] PUT /identity/v3/projects/1cd50a0a32ae481b82c64c706ebd1bda/users/9e74299d97bc48c59bed66ebea5d64e3/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:55.708232 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61b540-f861-4cbc-98a5-97504494103e None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:55.709280 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61b540-f861-4cbc-98a5-97504494103e None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:55.709280 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61b540-f861-4cbc-98a5-97504494103e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:55.741351 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-af61b540-f861-4cbc-98a5-97504494103e None None] MFA Rules not processed for user `9e74299d97bc48c59bed66ebea5d64e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:55.742753 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-af61b540-f861-4cbc-98a5-97504494103e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:55.823693 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1435/2845] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:55.836684 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf5fff4-2388-480a-ba75-3dd9b3ea82e9 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:55.838441 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-7cf5fff4-2388-480a-ba75-3dd9b3ea82e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:55.856605 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:55.912013 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf5fff4-2388-480a-ba75-3dd9b3ea82e9 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:55.916242 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df08d0f0-2661-4f03-ba12-a213702dec6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:55.917230 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08d0f0-2661-4f03-ba12-a213702dec6a service cinder] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:55.917536 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08d0f0-2661-4f03-ba12-a213702dec6a service cinder] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:55.917821 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df08d0f0-2661-4f03-ba12-a213702dec6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:55.928601 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df08d0f0-2661-4f03-ba12-a213702dec6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:55.930343 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df08d0f0-2661-4f03-ba12-a213702dec6a service cinder] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:55.941219 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1411/2846] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:31:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:31:59.327543 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a2eb8-1a4c-4b8f-84ea-9ca25a23ee13 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:59.328056 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a2eb8-1a4c-4b8f-84ea-9ca25a23ee13 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:59.328142 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a2eb8-1a4c-4b8f-84ea-9ca25a23ee13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:59.351485 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:31:59.360451 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-684a2eb8-1a4c-4b8f-84ea-9ca25a23ee13 None None] MFA Rules not processed for user `c338d0e7dce9404597cd0061e408091d`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:31:59.362307 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-684a2eb8-1a4c-4b8f-84ea-9ca25a23ee13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:59.435791 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1436/2847] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:31:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:31:59.440473 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df08d0f0-2661-4f03-ba12-a213702dec6a service cinder] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:59.442154 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-df08d0f0-2661-4f03-ba12-a213702dec6a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:31:59.521607 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df08d0f0-2661-4f03-ba12-a213702dec6a service cinder] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:59.530130 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f149bd-a9e1-4fd6-a203-03641271ee2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:59.530831 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f149bd-a9e1-4fd6-a203-03641271ee2c admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:59.531098 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f149bd-a9e1-4fd6-a203-03641271ee2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:59.531374 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f149bd-a9e1-4fd6-a203-03641271ee2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:59.532068 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f149bd-a9e1-4fd6-a203-03641271ee2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:59.534066 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f149bd-a9e1-4fd6-a203-03641271ee2c admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:59.573944 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1412/2848] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:31:59 2022] POST /identity/v3/projects => generated 371 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:59.577773 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684a2eb8-1a4c-4b8f-84ea-9ca25a23ee13 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:59.590810 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684a2eb8-1a4c-4b8f-84ea-9ca25a23ee13 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:59.659220 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a7bdb6-2a2f-47ea-9adb-818275ba2c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:59.659908 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a7bdb6-2a2f-47ea-9adb-818275ba2c3d admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:59.660184 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a7bdb6-2a2f-47ea-9adb-818275ba2c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:59.660457 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a7bdb6-2a2f-47ea-9adb-818275ba2c3d admin admin] PATH_INFO: `/v3/users` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:59.661187 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a7bdb6-2a2f-47ea-9adb-818275ba2c3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:59.663130 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a7bdb6-2a2f-47ea-9adb-818275ba2c3d admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:59.685442 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1437/2849] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:31:59 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:31:59.689821 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f149bd-a9e1-4fd6-a203-03641271ee2c admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:59.702245 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f149bd-a9e1-4fd6-a203-03641271ee2c admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:59.708636 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda876e8-76eb-4c91-964b-8196afbeb2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:59.709928 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda876e8-76eb-4c91-964b-8196afbeb2d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:59.710216 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda876e8-76eb-4c91-964b-8196afbeb2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:59.710497 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda876e8-76eb-4c91-964b-8196afbeb2d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:59.711251 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda876e8-76eb-4c91-964b-8196afbeb2d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:59.713092 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda876e8-76eb-4c91-964b-8196afbeb2d8 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:59.726116 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1413/2850] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:59.730345 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a7bdb6-2a2f-47ea-9adb-818275ba2c3d admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:59.742103 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a7bdb6-2a2f-47ea-9adb-818275ba2c3d admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:59.747848 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01648caf-ed78-4318-85eb-0331c789689f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:59.748858 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01648caf-ed78-4318-85eb-0331c789689f admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:59.749195 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01648caf-ed78-4318-85eb-0331c789689f admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:59.749476 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01648caf-ed78-4318-85eb-0331c789689f admin admin] PATH_INFO: `/v3/projects/407905b0b8764df0a86e1ac9f93b682c/users/539278709f6c4d8fb5bb744610531447/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:59.766437 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01648caf-ed78-4318-85eb-0331c789689f admin admin] RBAC: Authorizing `identity:create_grant(project_id=407905b0b8764df0a86e1ac9f93b682c, user_id=539278709f6c4d8fb5bb744610531447, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:59.769087 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01648caf-ed78-4318-85eb-0331c789689f admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:59.785353 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1438/2851] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:31:59 2022] PUT /identity/v3/projects/407905b0b8764df0a86e1ac9f93b682c/users/539278709f6c4d8fb5bb744610531447/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:31:59.789500 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda876e8-76eb-4c91-964b-8196afbeb2d8 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:59.800427 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda876e8-76eb-4c91-964b-8196afbeb2d8 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:59.864114 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c470f5e2-8e11-4eee-84c1-4dac1c122a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:59.864804 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470f5e2-8e11-4eee-84c1-4dac1c122a30 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:59.865130 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470f5e2-8e11-4eee-84c1-4dac1c122a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:59.865516 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470f5e2-8e11-4eee-84c1-4dac1c122a30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:59.866113 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c470f5e2-8e11-4eee-84c1-4dac1c122a30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:59.867715 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c470f5e2-8e11-4eee-84c1-4dac1c122a30 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:59.878783 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1414/2852] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:31:59 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:31:59.883065 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01648caf-ed78-4318-85eb-0331c789689f admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:31:59.894915 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01648caf-ed78-4318-85eb-0331c789689f admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:31:59.899796 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8e7fa-37ac-40b3-99d1-4e5c648d1644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:31:59.900743 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8e7fa-37ac-40b3-99d1-4e5c648d1644 admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:31:59.901066 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8e7fa-37ac-40b3-99d1-4e5c648d1644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:31:59.901482 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8e7fa-37ac-40b3-99d1-4e5c648d1644 admin admin] PATH_INFO: `/v3/projects/407905b0b8764df0a86e1ac9f93b682c/users/539278709f6c4d8fb5bb744610531447/roles/b4c524dd7bb44e94bd3b42d86e1cdf42` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:31:59.904184 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8e7fa-37ac-40b3-99d1-4e5c648d1644 admin admin] RBAC: Authorizing `identity:create_grant(project_id=407905b0b8764df0a86e1ac9f93b682c, user_id=539278709f6c4d8fb5bb744610531447, role_id=b4c524dd7bb44e94bd3b42d86e1cdf42)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:31:59.907037 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8e7fa-37ac-40b3-99d1-4e5c648d1644 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:31:59.919133 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1439/2853] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:31:59 2022] PUT /identity/v3/projects/407905b0b8764df0a86e1ac9f93b682c/users/539278709f6c4d8fb5bb744610531447/roles/b4c524dd7bb44e94bd3b42d86e1cdf42 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:32:01.917998 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c470f5e2-8e11-4eee-84c1-4dac1c122a30 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:01.946149 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c470f5e2-8e11-4eee-84c1-4dac1c122a30 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:02.020569 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:02.087706 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9521f9de-f8b1-4529-a19e-7d58659b4948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:02.088609 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521f9de-f8b1-4529-a19e-7d58659b4948 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:02.088895 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521f9de-f8b1-4529-a19e-7d58659b4948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:02.089308 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9521f9de-f8b1-4529-a19e-7d58659b4948 admin admin] PATH_INFO: `/v3/users/9e74299d97bc48c59bed66ebea5d64e3` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:02.090671 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9521f9de-f8b1-4529-a19e-7d58659b4948 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e74299d97bc48c59bed66ebea5d64e3)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:02.092667 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9521f9de-f8b1-4529-a19e-7d58659b4948 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:02.181538 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-9521f9de-f8b1-4529-a19e-7d58659b4948 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e74299d97bc48c59bed66ebea5d64e3'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:02.198144 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-9521f9de-f8b1-4529-a19e-7d58659b4948 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e74299d97bc48c59bed66ebea5d64e3'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:02.212750 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-9521f9de-f8b1-4529-a19e-7d58659b4948 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e74299d97bc48c59bed66ebea5d64e3'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:02.225311 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-9521f9de-f8b1-4529-a19e-7d58659b4948 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e74299d97bc48c59bed66ebea5d64e3'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:02.226391 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-9521f9de-f8b1-4529-a19e-7d58659b4948 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e74299d97bc48c59bed66ebea5d64e3'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:02.246278 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1415/2854] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:32:01 2022] DELETE /identity/v3/users/9e74299d97bc48c59bed66ebea5d64e3 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:32:02.251656 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcadbf3b-48aa-40ec-ab93-11855ff4830b None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:02.252125 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcadbf3b-48aa-40ec-ab93-11855ff4830b None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:02.252452 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcadbf3b-48aa-40ec-ab93-11855ff4830b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:02.285769 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-bcadbf3b-48aa-40ec-ab93-11855ff4830b None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:32:02.287324 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-bcadbf3b-48aa-40ec-ab93-11855ff4830b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:02.355868 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:02.377711 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1440/2855] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:32:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:32:02.394981 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9521f9de-f8b1-4529-a19e-7d58659b4948 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:02.397424 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-9521f9de-f8b1-4529-a19e-7d58659b4948 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:02.481277 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9521f9de-f8b1-4529-a19e-7d58659b4948 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:02.485645 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32808c70-0e7e-4dad-b577-234c95c014f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:02.486316 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32808c70-0e7e-4dad-b577-234c95c014f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:02.486596 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32808c70-0e7e-4dad-b577-234c95c014f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:02.486869 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32808c70-0e7e-4dad-b577-234c95c014f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:02.497982 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32808c70-0e7e-4dad-b577-234c95c014f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:02.499496 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32808c70-0e7e-4dad-b577-234c95c014f6 service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:02.511760 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1416/2856] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:32:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:32:03.455851 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcadbf3b-48aa-40ec-ab93-11855ff4830b None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:03.466662 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-bcadbf3b-48aa-40ec-ab93-11855ff4830b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:03.494230 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcadbf3b-48aa-40ec-ab93-11855ff4830b None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:03.510755 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f003d151-8c48-4333-9bb7-7b96a5368896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:03.512409 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003d151-8c48-4333-9bb7-7b96a5368896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:03.513016 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003d151-8c48-4333-9bb7-7b96a5368896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:03.513477 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003d151-8c48-4333-9bb7-7b96a5368896 admin admin] PATH_INFO: `/v3/projects/1cd50a0a32ae481b82c64c706ebd1bda` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:03.515412 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f003d151-8c48-4333-9bb7-7b96a5368896 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cd50a0a32ae481b82c64c706ebd1bda)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:03.518410 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f003d151-8c48-4333-9bb7-7b96a5368896 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:03.619057 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f003d151-8c48-4333-9bb7-7b96a5368896 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cd50a0a32ae481b82c64c706ebd1bda', 'request_id': 'req-f003d151-8c48-4333-9bb7-7b96a5368896'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:03.633966 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f003d151-8c48-4333-9bb7-7b96a5368896 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cd50a0a32ae481b82c64c706ebd1bda', 'request_id': 'req-f003d151-8c48-4333-9bb7-7b96a5368896'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:03.652497 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f003d151-8c48-4333-9bb7-7b96a5368896 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cd50a0a32ae481b82c64c706ebd1bda', 'request_id': 'req-f003d151-8c48-4333-9bb7-7b96a5368896'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:03.678520 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f003d151-8c48-4333-9bb7-7b96a5368896 admin admin] The token cache is being invalidate because project 1cd50a0a32ae481b82c64c706ebd1bda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:32:03.678818 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f003d151-8c48-4333-9bb7-7b96a5368896 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:03.679605 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-f003d151-8c48-4333-9bb7-7b96a5368896 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:03.686825 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e30cd-83be-4470-89b2-7d2ab7b7620a None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:03.687364 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e30cd-83be-4470-89b2-7d2ab7b7620a None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:03.687364 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190e30cd-83be-4470-89b2-7d2ab7b7620a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:03.689117 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1441/2857] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:32:03 2022] DELETE /identity/v3/projects/1cd50a0a32ae481b82c64c706ebd1bda => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:32:03.708500 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-190e30cd-83be-4470-89b2-7d2ab7b7620a None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:32:03.709795 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-190e30cd-83be-4470-89b2-7d2ab7b7620a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:03.801373 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:03.837914 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1417/2858] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:32:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:32:03.902081 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb72123-0fdf-45a9-900e-f606da1da614 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:03.902511 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb72123-0fdf-45a9-900e-f606da1da614 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:03.903074 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb72123-0fdf-45a9-900e-f606da1da614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:03.924341 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-5bb72123-0fdf-45a9-900e-f606da1da614 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:32:03.926156 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-5bb72123-0fdf-45a9-900e-f606da1da614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:03.939131 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190e30cd-83be-4470-89b2-7d2ab7b7620a None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:03.943596 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1442/2859] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:32:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:32:03.953846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190e30cd-83be-4470-89b2-7d2ab7b7620a None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:03.960330 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c22d776-9316-402b-bdf5-542ce0920bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:03.961670 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22d776-9316-402b-bdf5-542ce0920bb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:03.962248 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22d776-9316-402b-bdf5-542ce0920bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:03.962748 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22d776-9316-402b-bdf5-542ce0920bb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:03.963761 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c22d776-9316-402b-bdf5-542ce0920bb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80410) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 03:32:03.964333 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c22d776-9316-402b-bdf5-542ce0920bb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:03.966505 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c22d776-9316-402b-bdf5-542ce0920bb2 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:03.989999 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1418/2860] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 03:32:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 03:32:03.991973 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb72123-0fdf-45a9-900e-f606da1da614 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:04.012973 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb72123-0fdf-45a9-900e-f606da1da614 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:04.022738 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bbee9a-8360-41d1-85e0-d886db69f447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:04.023666 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bbee9a-8360-41d1-85e0-d886db69f447 admin admin] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:04.024071 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bbee9a-8360-41d1-85e0-d886db69f447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:04.024444 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bbee9a-8360-41d1-85e0-d886db69f447 admin admin] PATH_INFO: `/v3/projects` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:04.025283 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bbee9a-8360-41d1-85e0-d886db69f447 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:04.027468 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bbee9a-8360-41d1-85e0-d886db69f447 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:04.107892 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1443/2861] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 03:32:03 2022] POST /identity/v3/projects => generated 397 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:32:04.109705 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c22d776-9316-402b-bdf5-542ce0920bb2 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:04.127983 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c22d776-9316-402b-bdf5-542ce0920bb2 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:04.271085 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684edb8b-bb62-4821-8c03-7b761a11e03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:04.271791 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684edb8b-bb62-4821-8c03-7b761a11e03f admin admin] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:04.272066 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684edb8b-bb62-4821-8c03-7b761a11e03f admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:04.272341 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684edb8b-bb62-4821-8c03-7b761a11e03f admin admin] PATH_INFO: `/v3/users` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:04.273220 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684edb8b-bb62-4821-8c03-7b761a11e03f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:04.275181 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684edb8b-bb62-4821-8c03-7b761a11e03f admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:04.337059 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1419/2862] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 03:32:04 2022] POST /identity/v3/users => generated 299 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 03:32:04.340860 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bbee9a-8360-41d1-85e0-d886db69f447 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:04.358169 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bbee9a-8360-41d1-85e0-d886db69f447 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:04.370376 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab946ed-5d04-4e0e-a661-922d91947fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:04.373997 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab946ed-5d04-4e0e-a661-922d91947fc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:04.375878 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab946ed-5d04-4e0e-a661-922d91947fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:04.376318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab946ed-5d04-4e0e-a661-922d91947fc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:04.377481 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab946ed-5d04-4e0e-a661-922d91947fc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:04.379451 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab946ed-5d04-4e0e-a661-922d91947fc6 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:04.432842 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1444/2863] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:32:04 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:32:04.433719 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684edb8b-bb62-4821-8c03-7b761a11e03f admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:04.452223 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684edb8b-bb62-4821-8c03-7b761a11e03f admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:04.466059 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da311990-260c-43cf-a488-ef492cef22a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:04.466784 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da311990-260c-43cf-a488-ef492cef22a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:04.467189 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da311990-260c-43cf-a488-ef492cef22a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:04.467633 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da311990-260c-43cf-a488-ef492cef22a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:04.468434 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da311990-260c-43cf-a488-ef492cef22a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:04.470403 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da311990-260c-43cf-a488-ef492cef22a2 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:04.503800 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1420/2864] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 03:32:04 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 03:32:04.505355 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab946ed-5d04-4e0e-a661-922d91947fc6 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:04.525779 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab946ed-5d04-4e0e-a661-922d91947fc6 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:04.544679 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad218aa3-1512-40d9-a08f-7eba695ffbcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:04.545852 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad218aa3-1512-40d9-a08f-7eba695ffbcd admin admin] REQUEST_METHOD: `PUT` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:04.546351 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad218aa3-1512-40d9-a08f-7eba695ffbcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:04.546846 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad218aa3-1512-40d9-a08f-7eba695ffbcd admin admin] PATH_INFO: `/v3/projects/2b95cf6e51f941e093466830b92b2bb8/users/38e002ab56114f6d95d86f3077964166/roles/0129dd45ff5449f9957e8efdd01a55ed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:04.582433 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad218aa3-1512-40d9-a08f-7eba695ffbcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b95cf6e51f941e093466830b92b2bb8, user_id=38e002ab56114f6d95d86f3077964166, role_id=0129dd45ff5449f9957e8efdd01a55ed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:04.585436 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad218aa3-1512-40d9-a08f-7eba695ffbcd admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:04.612035 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1445/2865] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 03:32:04 2022] PUT /identity/v3/projects/2b95cf6e51f941e093466830b92b2bb8/users/38e002ab56114f6d95d86f3077964166/roles/0129dd45ff5449f9957e8efdd01a55ed => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:32:04.623743 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83010ae4-a22f-4135-8535-23891f1f599a None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:04.624036 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83010ae4-a22f-4135-8535-23891f1f599a None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:04.624296 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83010ae4-a22f-4135-8535-23891f1f599a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:04.679586 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-83010ae4-a22f-4135-8535-23891f1f599a None None] MFA Rules not processed for user `38e002ab56114f6d95d86f3077964166`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:32:04.681347 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-83010ae4-a22f-4135-8535-23891f1f599a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:04.817741 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1421/2866] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:32:04 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:32:04.830735 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad218aa3-1512-40d9-a08f-7eba695ffbcd admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:04.832551 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-ad218aa3-1512-40d9-a08f-7eba695ffbcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:04.958848 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad218aa3-1512-40d9-a08f-7eba695ffbcd admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:04.966017 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c9a61-aaf0-4667-a4dd-b314af3aab57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36dd9b94ceb74eb4a7e9b5bbd5e8d8ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:04.966979 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c9a61-aaf0-4667-a4dd-b314af3aab57 service cinder] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:04.967159 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c9a61-aaf0-4667-a4dd-b314af3aab57 service cinder] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:04.967447 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c9a61-aaf0-4667-a4dd-b314af3aab57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:05.005616 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94c9a61-aaf0-4667-a4dd-b314af3aab57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:05.013811 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94c9a61-aaf0-4667-a4dd-b314af3aab57 service cinder] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:05.050524 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1446/2867] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:32:04 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:32:05.773046 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d734d-a40c-415f-b4c2-a4ad2279daf6 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:05.774172 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d734d-a40c-415f-b4c2-a4ad2279daf6 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:05.774645 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286d734d-a40c-415f-b4c2-a4ad2279daf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:05.835290 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-286d734d-a40c-415f-b4c2-a4ad2279daf6 None None] MFA Rules not processed for user `539278709f6c4d8fb5bb744610531447`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:32:05.836596 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-286d734d-a40c-415f-b4c2-a4ad2279daf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:05.893069 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:05.939822 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1422/2868] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:32:05 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:32:05.962689 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c9a61-aaf0-4667-a4dd-b314af3aab57 service cinder] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:05.964161 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-e94c9a61-aaf0-4667-a4dd-b314af3aab57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:05.992878 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:06.062440 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94c9a61-aaf0-4667-a4dd-b314af3aab57 service cinder] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:06.067809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce3815-1867-49c0-b4d3-d0044fe70668 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:06.068466 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce3815-1867-49c0-b4d3-d0044fe70668 service nova] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:06.068748 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce3815-1867-49c0-b4d3-d0044fe70668 service nova] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:06.069056 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fce3815-1867-49c0-b4d3-d0044fe70668 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:06.070834 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-1fce3815-1867-49c0-b4d3-d0044fe70668 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:06.149198 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce3815-1867-49c0-b4d3-d0044fe70668 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:06.151025 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fce3815-1867-49c0-b4d3-d0044fe70668 service nova] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:06.162274 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1447/2869] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:32:05 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:32:07.345413 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4fa46-f1fc-4ed4-88d8-cc6b9be287b5 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:07.346321 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4fa46-f1fc-4ed4-88d8-cc6b9be287b5 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:07.346599 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4fa46-f1fc-4ed4-88d8-cc6b9be287b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:07.365525 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-64d4fa46-f1fc-4ed4-88d8-cc6b9be287b5 None None] MFA Rules not processed for user `c338d0e7dce9404597cd0061e408091d`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:32:07.366987 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-64d4fa46-f1fc-4ed4-88d8-cc6b9be287b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:07.375878 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1423/2870] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:32:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:32:11.384351 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce3815-1867-49c0-b4d3-d0044fe70668 service nova] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:11.386368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-1fce3815-1867-49c0-b4d3-d0044fe70668 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:11.458883 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fce3815-1867-49c0-b4d3-d0044fe70668 service nova] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:11.463407 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7688e-f929-4843-a897-08612b04d69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:11.464317 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7688e-f929-4843-a897-08612b04d69c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:11.464596 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7688e-f929-4843-a897-08612b04d69c admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:11.464873 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7688e-f929-4843-a897-08612b04d69c admin admin] PATH_INFO: `/v3/users/0d051848c7014e6495111b4f13afc000` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:11.466113 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad7688e-f929-4843-a897-08612b04d69c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d051848c7014e6495111b4f13afc000)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:11.468234 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad7688e-f929-4843-a897-08612b04d69c admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:11.524769 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-fad7688e-f929-4843-a897-08612b04d69c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d051848c7014e6495111b4f13afc000'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:11.525566 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-fad7688e-f929-4843-a897-08612b04d69c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d051848c7014e6495111b4f13afc000'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:11.536353 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-fad7688e-f929-4843-a897-08612b04d69c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d051848c7014e6495111b4f13afc000'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:11.537337 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-fad7688e-f929-4843-a897-08612b04d69c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d051848c7014e6495111b4f13afc000'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:11.538900 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:11.552559 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-fad7688e-f929-4843-a897-08612b04d69c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d051848c7014e6495111b4f13afc000'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:11.564880 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1448/2871] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:32:11 2022] DELETE /identity/v3/users/0d051848c7014e6495111b4f13afc000 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:32:11.569705 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba5963-11f2-40f7-95a3-c44ddd2ec25b None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:11.570225 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba5963-11f2-40f7-95a3-c44ddd2ec25b None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:11.570328 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba5963-11f2-40f7-95a3-c44ddd2ec25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:11.589891 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-feba5963-11f2-40f7-95a3-c44ddd2ec25b None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:32:11.591114 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-feba5963-11f2-40f7-95a3-c44ddd2ec25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:11.663667 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1424/2872] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:32:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:32:11.674786 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7688e-f929-4843-a897-08612b04d69c admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:11.676336 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-fad7688e-f929-4843-a897-08612b04d69c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:11.760262 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7688e-f929-4843-a897-08612b04d69c admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:11.764806 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d2f8b-25be-4e2e-9aa9-767abdf29339 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:11.765509 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d2f8b-25be-4e2e-9aa9-767abdf29339 service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:11.765809 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d2f8b-25be-4e2e-9aa9-767abdf29339 service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:11.766074 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d2f8b-25be-4e2e-9aa9-767abdf29339 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:11.776682 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194d2f8b-25be-4e2e-9aa9-767abdf29339 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:11.778163 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194d2f8b-25be-4e2e-9aa9-767abdf29339 service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:11.788403 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1449/2873] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:32:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:32:12.110337 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feba5963-11f2-40f7-95a3-c44ddd2ec25b None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:12.112268 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-feba5963-11f2-40f7-95a3-c44ddd2ec25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:12.125195 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feba5963-11f2-40f7-95a3-c44ddd2ec25b None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:12.131463 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc247b34-dfc0-4562-a62e-52cc4c3bad2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:12.132441 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc247b34-dfc0-4562-a62e-52cc4c3bad2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:12.132725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc247b34-dfc0-4562-a62e-52cc4c3bad2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:12.133118 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc247b34-dfc0-4562-a62e-52cc4c3bad2f admin admin] PATH_INFO: `/v3/projects/5dd967b52989488b91e535ac5f8a93d9` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:12.134273 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc247b34-dfc0-4562-a62e-52cc4c3bad2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dd967b52989488b91e535ac5f8a93d9)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:12.137191 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc247b34-dfc0-4562-a62e-52cc4c3bad2f admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:12.188674 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-cc247b34-dfc0-4562-a62e-52cc4c3bad2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dd967b52989488b91e535ac5f8a93d9', 'request_id': 'req-cc247b34-dfc0-4562-a62e-52cc4c3bad2f'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:12.195868 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-cc247b34-dfc0-4562-a62e-52cc4c3bad2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dd967b52989488b91e535ac5f8a93d9', 'request_id': 'req-cc247b34-dfc0-4562-a62e-52cc4c3bad2f'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:12.201996 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-cc247b34-dfc0-4562-a62e-52cc4c3bad2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dd967b52989488b91e535ac5f8a93d9', 'request_id': 'req-cc247b34-dfc0-4562-a62e-52cc4c3bad2f'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:12.218392 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-cc247b34-dfc0-4562-a62e-52cc4c3bad2f admin admin] The token cache is being invalidate because project 5dd967b52989488b91e535ac5f8a93d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:32:12.218680 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-cc247b34-dfc0-4562-a62e-52cc4c3bad2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:12.219441 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-cc247b34-dfc0-4562-a62e-52cc4c3bad2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:12.221571 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1425/2874] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:32:12 2022] DELETE /identity/v3/projects/5dd967b52989488b91e535ac5f8a93d9 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:32:13.622490 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d2f8b-25be-4e2e-9aa9-767abdf29339 service neutron] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:13.624111 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-194d2f8b-25be-4e2e-9aa9-767abdf29339 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:13.638548 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d2f8b-25be-4e2e-9aa9-767abdf29339 service neutron] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:13.704003 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450892eb-bfc7-4cba-8fd5-f917144f6e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:13.704889 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450892eb-bfc7-4cba-8fd5-f917144f6e6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:13.705210 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450892eb-bfc7-4cba-8fd5-f917144f6e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:13.705485 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450892eb-bfc7-4cba-8fd5-f917144f6e6f admin admin] PATH_INFO: `/v3/users/38e002ab56114f6d95d86f3077964166` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:13.706795 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450892eb-bfc7-4cba-8fd5-f917144f6e6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=38e002ab56114f6d95d86f3077964166)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:13.708825 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450892eb-bfc7-4cba-8fd5-f917144f6e6f admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:13.765749 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-450892eb-bfc7-4cba-8fd5-f917144f6e6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38e002ab56114f6d95d86f3077964166'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:13.766760 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-450892eb-bfc7-4cba-8fd5-f917144f6e6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38e002ab56114f6d95d86f3077964166'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:13.778368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-450892eb-bfc7-4cba-8fd5-f917144f6e6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38e002ab56114f6d95d86f3077964166'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:13.779465 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-450892eb-bfc7-4cba-8fd5-f917144f6e6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38e002ab56114f6d95d86f3077964166'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:13.793264 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-450892eb-bfc7-4cba-8fd5-f917144f6e6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38e002ab56114f6d95d86f3077964166'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:13.797820 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:13.807124 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1450/2875] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:32:13 2022] DELETE /identity/v3/users/38e002ab56114f6d95d86f3077964166 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:32:13.811954 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b0fa52-46ad-46dd-8469-924c3a330021 None None] REQUEST_METHOD: `POST` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:13.812435 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b0fa52-46ad-46dd-8469-924c3a330021 None None] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:13.812563 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b0fa52-46ad-46dd-8469-924c3a330021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:13.832243 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.auth.core [None req-74b0fa52-46ad-46dd-8469-924c3a330021 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80410) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:32:13.833494 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-74b0fa52-46ad-46dd-8469-924c3a330021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:13.876629 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:13.897155 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1426/2876] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:32:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:32:13.907676 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450892eb-bfc7-4cba-8fd5-f917144f6e6f admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:13.909450 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-450892eb-bfc7-4cba-8fd5-f917144f6e6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:13.975987 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450892eb-bfc7-4cba-8fd5-f917144f6e6f admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:13.976231 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b0fa52-46ad-46dd-8469-924c3a330021 None None] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:13.977847 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-74b0fa52-46ad-46dd-8469-924c3a330021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:13.979821 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f40230-1835-4b07-9f4d-9ac4b53c1d18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:13.980475 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f40230-1835-4b07-9f4d-9ac4b53c1d18 service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:13.980751 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f40230-1835-4b07-9f4d-9ac4b53c1d18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:13.981042 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f40230-1835-4b07-9f4d-9ac4b53c1d18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:13.990430 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b0fa52-46ad-46dd-8469-924c3a330021 None None] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:13.992845 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f40230-1835-4b07-9f4d-9ac4b53c1d18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:13.994431 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f40230-1835-4b07-9f4d-9ac4b53c1d18 service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:13.995294 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb1438-2895-4925-b8e4-1f67353c1530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:13.996189 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb1438-2895-4925-b8e4-1f67353c1530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:13.996473 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb1438-2895-4925-b8e4-1f67353c1530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:13.996745 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb1438-2895-4925-b8e4-1f67353c1530 admin admin] PATH_INFO: `/v3/users/b8b2deb6f91548d8b831ce892def114c` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:13.998109 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfb1438-2895-4925-b8e4-1f67353c1530 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8b2deb6f91548d8b831ce892def114c)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:14.000032 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfb1438-2895-4925-b8e4-1f67353c1530 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:14.005381 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1451/2877] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:32:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:32:14.049920 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-6bfb1438-2895-4925-b8e4-1f67353c1530 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8b2deb6f91548d8b831ce892def114c'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:14.064014 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-6bfb1438-2895-4925-b8e4-1f67353c1530 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8b2deb6f91548d8b831ce892def114c'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:14.073063 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-6bfb1438-2895-4925-b8e4-1f67353c1530 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8b2deb6f91548d8b831ce892def114c'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:14.080937 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-6bfb1438-2895-4925-b8e4-1f67353c1530 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8b2deb6f91548d8b831ce892def114c'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:14.082326 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-6bfb1438-2895-4925-b8e4-1f67353c1530 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8b2deb6f91548d8b831ce892def114c'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:14.090325 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1427/2878] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:32:13 2022] DELETE /identity/v3/users/b8b2deb6f91548d8b831ce892def114c => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:32:14.095557 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae014bb6-e338-4212-9a02-1b735197da87 None None] REQUEST_METHOD: `POST` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:14.096078 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae014bb6-e338-4212-9a02-1b735197da87 None None] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:14.096544 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae014bb6-e338-4212-9a02-1b735197da87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:14.119780 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.auth.core [None req-ae014bb6-e338-4212-9a02-1b735197da87 None None] MFA Rules not processed for user `4a96d7acf93e46c7ac5700560db36dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80409) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 03:32:14.121044 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-ae014bb6-e338-4212-9a02-1b735197da87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:14.196268 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1452/2879] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 03:32:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 03:32:14.208620 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb1438-2895-4925-b8e4-1f67353c1530 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:14.210109 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-6bfb1438-2895-4925-b8e4-1f67353c1530 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:14.279238 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb1438-2895-4925-b8e4-1f67353c1530 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:14.283081 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ea865-cce8-4234-8267-51064cdc63b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:14.283747 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ea865-cce8-4234-8267-51064cdc63b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:14.284022 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ea865-cce8-4234-8267-51064cdc63b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:14.284297 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ea865-cce8-4234-8267-51064cdc63b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:14.295995 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ea865-cce8-4234-8267-51064cdc63b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:14.297502 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ea865-cce8-4234-8267-51064cdc63b0 service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:14.308932 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1428/2880] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:32:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:32:14.311601 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae014bb6-e338-4212-9a02-1b735197da87 None None] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:14.313097 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-ae014bb6-e338-4212-9a02-1b735197da87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:14.325197 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae014bb6-e338-4212-9a02-1b735197da87 None None] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:14.335687 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034784ac-02ce-40a1-b0b8-ced7e4a3e717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:14.336585 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034784ac-02ce-40a1-b0b8-ced7e4a3e717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:14.336871 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034784ac-02ce-40a1-b0b8-ced7e4a3e717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:14.337180 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034784ac-02ce-40a1-b0b8-ced7e4a3e717 admin admin] PATH_INFO: `/v3/projects/2b95cf6e51f941e093466830b92b2bb8` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:14.338387 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034784ac-02ce-40a1-b0b8-ced7e4a3e717 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b95cf6e51f941e093466830b92b2bb8)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:14.340698 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034784ac-02ce-40a1-b0b8-ced7e4a3e717 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:14.393719 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-034784ac-02ce-40a1-b0b8-ced7e4a3e717 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b95cf6e51f941e093466830b92b2bb8', 'request_id': 'req-034784ac-02ce-40a1-b0b8-ced7e4a3e717'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:14.400974 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-034784ac-02ce-40a1-b0b8-ced7e4a3e717 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b95cf6e51f941e093466830b92b2bb8', 'request_id': 'req-034784ac-02ce-40a1-b0b8-ced7e4a3e717'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:14.410676 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-034784ac-02ce-40a1-b0b8-ced7e4a3e717 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b95cf6e51f941e093466830b92b2bb8', 'request_id': 'req-034784ac-02ce-40a1-b0b8-ced7e4a3e717'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:14.424294 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-034784ac-02ce-40a1-b0b8-ced7e4a3e717 admin admin] The token cache is being invalidate because project 2b95cf6e51f941e093466830b92b2bb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:32:14.424588 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-034784ac-02ce-40a1-b0b8-ced7e4a3e717 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:14.425484 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-034784ac-02ce-40a1-b0b8-ced7e4a3e717 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:14.427219 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1453/2881] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:32:14 2022] DELETE /identity/v3/projects/2b95cf6e51f941e093466830b92b2bb8 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:32:14.635361 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ea865-cce8-4234-8267-51064cdc63b0 service neutron] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:14.637513 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-d86ea865-cce8-4234-8267-51064cdc63b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:14.661415 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ea865-cce8-4234-8267-51064cdc63b0 service neutron] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:14.734585 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6325b0-b944-4cc1-8f1f-bd93597e0973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:14.735460 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6325b0-b944-4cc1-8f1f-bd93597e0973 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:14.735736 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6325b0-b944-4cc1-8f1f-bd93597e0973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:14.736008 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6325b0-b944-4cc1-8f1f-bd93597e0973 admin admin] PATH_INFO: `/v3/projects/14f492b5b70c4736ab571b5b3d331bf8` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:14.737209 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6325b0-b944-4cc1-8f1f-bd93597e0973 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14f492b5b70c4736ab571b5b3d331bf8)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:14.739147 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6325b0-b944-4cc1-8f1f-bd93597e0973 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:14.741445 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:14.790181 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-de6325b0-b944-4cc1-8f1f-bd93597e0973 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14f492b5b70c4736ab571b5b3d331bf8', 'request_id': 'req-de6325b0-b944-4cc1-8f1f-bd93597e0973'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:14.796767 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-de6325b0-b944-4cc1-8f1f-bd93597e0973 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14f492b5b70c4736ab571b5b3d331bf8', 'request_id': 'req-de6325b0-b944-4cc1-8f1f-bd93597e0973'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:14.803230 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-de6325b0-b944-4cc1-8f1f-bd93597e0973 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14f492b5b70c4736ab571b5b3d331bf8', 'request_id': 'req-de6325b0-b944-4cc1-8f1f-bd93597e0973'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:14.817965 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-de6325b0-b944-4cc1-8f1f-bd93597e0973 admin admin] The token cache is being invalidate because project 14f492b5b70c4736ab571b5b3d331bf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:32:14.818591 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-de6325b0-b944-4cc1-8f1f-bd93597e0973 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:14.818861 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-de6325b0-b944-4cc1-8f1f-bd93597e0973 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:32:14.820437 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1429/2882] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:32:14 2022] DELETE /identity/v3/projects/14f492b5b70c4736ab571b5b3d331bf8 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:32:24.420319 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:24.814887 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=23,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:32:43.294209 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034784ac-02ce-40a1-b0b8-ced7e4a3e717 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:32:43.295837 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-034784ac-02ce-40a1-b0b8-ced7e4a3e717 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:43.363558 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034784ac-02ce-40a1-b0b8-ced7e4a3e717 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:32:43.418700 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e568da1d-179d-4804-a43d-08c30452a3e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:32:43.419350 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568da1d-179d-4804-a43d-08c30452a3e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:32:43.419630 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568da1d-179d-4804-a43d-08c30452a3e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:32:43.419899 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568da1d-179d-4804-a43d-08c30452a3e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:32:43.421991 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-e568da1d-179d-4804-a43d-08c30452a3e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:32:43.495102 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e568da1d-179d-4804-a43d-08c30452a3e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:32:43.496468 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e568da1d-179d-4804-a43d-08c30452a3e4 service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:32:43.507031 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1454/2883] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:32:43 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:32:53.502838 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=70 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:09.506051 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6325b0-b944-4cc1-8f1f-bd93597e0973 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:09.507584 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-de6325b0-b944-4cc1-8f1f-bd93597e0973 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:33:09.520598 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6325b0-b944-4cc1-8f1f-bd93597e0973 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:09.524574 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759266f0-f026-4b8e-b319-cbfec44ddb48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:09.525295 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759266f0-f026-4b8e-b319-cbfec44ddb48 service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:09.525568 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759266f0-f026-4b8e-b319-cbfec44ddb48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:09.525840 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759266f0-f026-4b8e-b319-cbfec44ddb48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:09.528244 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-759266f0-f026-4b8e-b319-cbfec44ddb48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:33:09.593216 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759266f0-f026-4b8e-b319-cbfec44ddb48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:09.594650 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759266f0-f026-4b8e-b319-cbfec44ddb48 service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:09.604990 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1430/2884] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:33:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:33:16.075312 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e568da1d-179d-4804-a43d-08c30452a3e4 service neutron] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:16.077420 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-e568da1d-179d-4804-a43d-08c30452a3e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:33:16.092521 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e568da1d-179d-4804-a43d-08c30452a3e4 service neutron] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:16.097197 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89ef83d-b404-45da-b836-69309b16fdb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:16.098089 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ef83d-b404-45da-b836-69309b16fdb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:16.098515 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ef83d-b404-45da-b836-69309b16fdb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:16.098791 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ef83d-b404-45da-b836-69309b16fdb8 admin admin] PATH_INFO: `/v3/users/01be6ea7136e45aead352146193f05ed` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:16.100039 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89ef83d-b404-45da-b836-69309b16fdb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01be6ea7136e45aead352146193f05ed)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:16.102516 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89ef83d-b404-45da-b836-69309b16fdb8 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:16.159006 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-c89ef83d-b404-45da-b836-69309b16fdb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01be6ea7136e45aead352146193f05ed'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:33:16.159716 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-c89ef83d-b404-45da-b836-69309b16fdb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01be6ea7136e45aead352146193f05ed'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:33:16.169521 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-c89ef83d-b404-45da-b836-69309b16fdb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01be6ea7136e45aead352146193f05ed'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:33:16.170244 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-c89ef83d-b404-45da-b836-69309b16fdb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01be6ea7136e45aead352146193f05ed'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:33:16.184390 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-c89ef83d-b404-45da-b836-69309b16fdb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01be6ea7136e45aead352146193f05ed'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:33:16.198501 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1455/2885] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:33:16 2022] DELETE /identity/v3/users/01be6ea7136e45aead352146193f05ed => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 03:33:16.400219 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759266f0-f026-4b8e-b319-cbfec44ddb48 service neutron] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:16.411990 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759266f0-f026-4b8e-b319-cbfec44ddb48 service neutron] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:16.470814 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca8a704-fc2e-457f-9f41-d36b9069f9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:16.471719 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8a704-fc2e-457f-9f41-d36b9069f9ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:16.472000 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8a704-fc2e-457f-9f41-d36b9069f9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:16.472276 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8a704-fc2e-457f-9f41-d36b9069f9ce admin admin] PATH_INFO: `/v3/projects/d5bef1b49d424299864523887eb72b3c` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:16.473529 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca8a704-fc2e-457f-9f41-d36b9069f9ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5bef1b49d424299864523887eb72b3c)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:16.475482 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca8a704-fc2e-457f-9f41-d36b9069f9ce admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:16.525651 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-aca8a704-fc2e-457f-9f41-d36b9069f9ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5bef1b49d424299864523887eb72b3c', 'request_id': 'req-aca8a704-fc2e-457f-9f41-d36b9069f9ce'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:33:16.531914 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-aca8a704-fc2e-457f-9f41-d36b9069f9ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5bef1b49d424299864523887eb72b3c', 'request_id': 'req-aca8a704-fc2e-457f-9f41-d36b9069f9ce'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:33:16.537541 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-aca8a704-fc2e-457f-9f41-d36b9069f9ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5bef1b49d424299864523887eb72b3c', 'request_id': 'req-aca8a704-fc2e-457f-9f41-d36b9069f9ce'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:33:16.552485 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-aca8a704-fc2e-457f-9f41-d36b9069f9ce admin admin] The token cache is being invalidate because project d5bef1b49d424299864523887eb72b3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:33:16.552722 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-aca8a704-fc2e-457f-9f41-d36b9069f9ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:33:16.553431 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-aca8a704-fc2e-457f-9f41-d36b9069f9ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:33:16.555115 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1431/2886] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:33:16 2022] DELETE /identity/v3/projects/d5bef1b49d424299864523887eb72b3c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:33:16.559434 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89ef83d-b404-45da-b836-69309b16fdb8 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:16.560986 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-c89ef83d-b404-45da-b836-69309b16fdb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:33:16.574860 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89ef83d-b404-45da-b836-69309b16fdb8 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:16.633824 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a6fed-9c96-4f8f-99c1-fa3d0f186270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:16.634723 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a6fed-9c96-4f8f-99c1-fa3d0f186270 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:16.635001 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a6fed-9c96-4f8f-99c1-fa3d0f186270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:16.635251 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a6fed-9c96-4f8f-99c1-fa3d0f186270 admin admin] PATH_INFO: `/v3/users/627f0217ac8344318fb58db6dc918649` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:16.636528 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8a6fed-9c96-4f8f-99c1-fa3d0f186270 admin admin] RBAC: Authorizing `identity:delete_user(user_id=627f0217ac8344318fb58db6dc918649)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:16.638909 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8a6fed-9c96-4f8f-99c1-fa3d0f186270 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:16.700182 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-2c8a6fed-9c96-4f8f-99c1-fa3d0f186270 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '627f0217ac8344318fb58db6dc918649'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:33:16.700842 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-2c8a6fed-9c96-4f8f-99c1-fa3d0f186270 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '627f0217ac8344318fb58db6dc918649'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:33:16.709793 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-2c8a6fed-9c96-4f8f-99c1-fa3d0f186270 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '627f0217ac8344318fb58db6dc918649'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:33:16.710624 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-2c8a6fed-9c96-4f8f-99c1-fa3d0f186270 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '627f0217ac8344318fb58db6dc918649'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:33:16.727510 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-2c8a6fed-9c96-4f8f-99c1-fa3d0f186270 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '627f0217ac8344318fb58db6dc918649'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:33:16.730204 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:16.743804 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1456/2887] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:33:16 2022] DELETE /identity/v3/users/627f0217ac8344318fb58db6dc918649 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:33:16.932041 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca8a704-fc2e-457f-9f41-d36b9069f9ce admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:33:16.933656 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-aca8a704-fc2e-457f-9f41-d36b9069f9ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:33:16.948703 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca8a704-fc2e-457f-9f41-d36b9069f9ce admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:33:16.973611 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:17.018556 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313107a1-5e90-4457-907c-58dab8aa4c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:33:17.019444 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313107a1-5e90-4457-907c-58dab8aa4c1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:33:17.019725 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313107a1-5e90-4457-907c-58dab8aa4c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:33:17.019999 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313107a1-5e90-4457-907c-58dab8aa4c1c admin admin] PATH_INFO: `/v3/projects/1021e1aa0d4143d8a6513d484f48f601` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:33:17.021209 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313107a1-5e90-4457-907c-58dab8aa4c1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1021e1aa0d4143d8a6513d484f48f601)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:33:17.023921 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313107a1-5e90-4457-907c-58dab8aa4c1c admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:33:17.076743 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-313107a1-5e90-4457-907c-58dab8aa4c1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1021e1aa0d4143d8a6513d484f48f601', 'request_id': 'req-313107a1-5e90-4457-907c-58dab8aa4c1c'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:33:17.083812 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-313107a1-5e90-4457-907c-58dab8aa4c1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1021e1aa0d4143d8a6513d484f48f601', 'request_id': 'req-313107a1-5e90-4457-907c-58dab8aa4c1c'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:33:17.091215 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-313107a1-5e90-4457-907c-58dab8aa4c1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1021e1aa0d4143d8a6513d484f48f601', 'request_id': 'req-313107a1-5e90-4457-907c-58dab8aa4c1c'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:33:17.106759 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-313107a1-5e90-4457-907c-58dab8aa4c1c admin admin] The token cache is being invalidate because project 1021e1aa0d4143d8a6513d484f48f601 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80410) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:33:17.107369 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-313107a1-5e90-4457-907c-58dab8aa4c1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:33:17.108063 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-313107a1-5e90-4457-907c-58dab8aa4c1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:33:17.109704 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1432/2888] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:33:16 2022] DELETE /identity/v3/projects/1021e1aa0d4143d8a6513d484f48f601 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:33:26.745051 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=4 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:33:27.103788 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:34:24.761591 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a6fed-9c96-4f8f-99c1-fa3d0f186270 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:34:24.763291 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-2c8a6fed-9c96-4f8f-99c1-fa3d0f186270 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:34:24.824129 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a6fed-9c96-4f8f-99c1-fa3d0f186270 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:34:24.892683 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e0feb-ef07-4181-b98e-b91e6bb92804 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:34:24.893364 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e0feb-ef07-4181-b98e-b91e6bb92804 service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:34:24.893641 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e0feb-ef07-4181-b98e-b91e6bb92804 service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:34:24.893910 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e0feb-ef07-4181-b98e-b91e6bb92804 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:34:24.895878 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-185e0feb-ef07-4181-b98e-b91e6bb92804 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:34:24.905763 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e0feb-ef07-4181-b98e-b91e6bb92804 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:34:24.907199 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e0feb-ef07-4181-b98e-b91e6bb92804 service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:34:24.915421 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1457/2889] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:34:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:34:34.911794 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=46 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:34:38.043776 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313107a1-5e90-4457-907c-58dab8aa4c1c admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:34:38.045994 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-313107a1-5e90-4457-907c-58dab8aa4c1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:34:38.062933 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313107a1-5e90-4457-907c-58dab8aa4c1c admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:34:38.071112 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c697f-81ed-4856-bbc9-7466ad07ecda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:34:38.071765 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c697f-81ed-4856-bbc9-7466ad07ecda service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:34:38.072047 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c697f-81ed-4856-bbc9-7466ad07ecda service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:34:38.072323 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c697f-81ed-4856-bbc9-7466ad07ecda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:34:38.074207 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-371c697f-81ed-4856-bbc9-7466ad07ecda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:34:38.165808 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371c697f-81ed-4856-bbc9-7466ad07ecda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:34:38.167196 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371c697f-81ed-4856-bbc9-7466ad07ecda service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:34:38.183186 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1433/2890] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:34:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:34:39.396557 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e0feb-ef07-4181-b98e-b91e6bb92804 service neutron] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:34:39.398307 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-185e0feb-ef07-4181-b98e-b91e6bb92804 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:34:39.409800 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e0feb-ef07-4181-b98e-b91e6bb92804 service neutron] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:34:39.414343 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ba653a-8385-4590-bcda-8bbef25fe6c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:34:39.415619 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba653a-8385-4590-bcda-8bbef25fe6c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:34:39.415925 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba653a-8385-4590-bcda-8bbef25fe6c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:34:39.416852 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba653a-8385-4590-bcda-8bbef25fe6c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:34:39.419778 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-30ba653a-8385-4590-bcda-8bbef25fe6c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:34:39.436255 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ba653a-8385-4590-bcda-8bbef25fe6c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:34:39.437730 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ba653a-8385-4590-bcda-8bbef25fe6c7 service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:34:39.449678 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1458/2891] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:34:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:34:48.174114 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=34 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:34:49.444121 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=6 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:34:51.648250 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c697f-81ed-4856-bbc9-7466ad07ecda service neutron] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:34:51.657443 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c697f-81ed-4856-bbc9-7466ad07ecda service neutron] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:34:51.662051 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459409b-45fe-4633-9c87-940ac747a179 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:34:51.662701 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459409b-45fe-4633-9c87-940ac747a179 service neutron] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:34:51.662981 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459409b-45fe-4633-9c87-940ac747a179 service neutron] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:34:51.663248 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6459409b-45fe-4633-9c87-940ac747a179 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:34:51.669368 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-6459409b-45fe-4633-9c87-940ac747a179 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:34:51.679702 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6459409b-45fe-4633-9c87-940ac747a179 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:34:51.681182 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6459409b-45fe-4633-9c87-940ac747a179 service neutron] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:34:51.689504 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1434/2892] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:34:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:34:51.764033 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ba653a-8385-4590-bcda-8bbef25fe6c7 service neutron] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:34:51.774011 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ba653a-8385-4590-bcda-8bbef25fe6c7 service neutron] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:34:51.778010 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da8316-c813-4119-ba03-d67ccaa7cba7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ee30c0d6d694ee8945800157a9fc5b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:34:51.778679 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da8316-c813-4119-ba03-d67ccaa7cba7 service neutron] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:34:51.778992 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da8316-c813-4119-ba03-d67ccaa7cba7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:34:51.779239 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da8316-c813-4119-ba03-d67ccaa7cba7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:34:51.781112 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-36da8316-c813-4119-ba03-d67ccaa7cba7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:34:51.795318 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da8316-c813-4119-ba03-d67ccaa7cba7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:34:51.796762 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da8316-c813-4119-ba03-d67ccaa7cba7 service neutron] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:34:51.808290 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1459/2893] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:34:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:34:51.872984 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459409b-45fe-4633-9c87-940ac747a179 service neutron] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:34:51.874574 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-6459409b-45fe-4633-9c87-940ac747a179 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:34:51.958031 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6459409b-45fe-4633-9c87-940ac747a179 service neutron] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:34:51.964528 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f201a6d6-7134-47d5-a3fb-d54b408e7aa5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30963f2f464e4f54a8aea0175370a235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:34:51.965211 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201a6d6-7134-47d5-a3fb-d54b408e7aa5 service placement] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:34:51.965489 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201a6d6-7134-47d5-a3fb-d54b408e7aa5 service placement] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:34:51.965759 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201a6d6-7134-47d5-a3fb-d54b408e7aa5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:34:51.967962 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-f201a6d6-7134-47d5-a3fb-d54b408e7aa5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:34:51.980097 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f201a6d6-7134-47d5-a3fb-d54b408e7aa5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:34:51.981748 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f201a6d6-7134-47d5-a3fb-d54b408e7aa5 service placement] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:34:51.991783 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1435/2894] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:34:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:34:53.845270 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da8316-c813-4119-ba03-d67ccaa7cba7 service neutron] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:34:53.847647 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-36da8316-c813-4119-ba03-d67ccaa7cba7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:34:53.864859 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da8316-c813-4119-ba03-d67ccaa7cba7 service neutron] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:34:53.871493 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1f7c06-e9a6-4187-881f-219c3b9a0b31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:34:53.872155 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f7c06-e9a6-4187-881f-219c3b9a0b31 service nova] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:34:53.872438 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f7c06-e9a6-4187-881f-219c3b9a0b31 service nova] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:34:53.872734 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f7c06-e9a6-4187-881f-219c3b9a0b31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:34:53.874527 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-fa1f7c06-e9a6-4187-881f-219c3b9a0b31 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:34:53.888448 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1f7c06-e9a6-4187-881f-219c3b9a0b31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:34:53.891165 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1f7c06-e9a6-4187-881f-219c3b9a0b31 service nova] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:34:53.903709 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1460/2895] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:34:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:35:01.987408 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=38 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:35:03.898643 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=12 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:15.358351 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f201a6d6-7134-47d5-a3fb-d54b408e7aa5 service placement] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:15.359948 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-f201a6d6-7134-47d5-a3fb-d54b408e7aa5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:15.445445 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f201a6d6-7134-47d5-a3fb-d54b408e7aa5 service placement] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:15.452439 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18c2ba9-f013-4d10-b4e8-d155856a94c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:15.454157 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c2ba9-f013-4d10-b4e8-d155856a94c2 service nova] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:15.455153 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c2ba9-f013-4d10-b4e8-d155856a94c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:15.455438 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c2ba9-f013-4d10-b4e8-d155856a94c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:15.457526 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-c18c2ba9-f013-4d10-b4e8-d155856a94c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:15.472544 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18c2ba9-f013-4d10-b4e8-d155856a94c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:15.477097 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18c2ba9-f013-4d10-b4e8-d155856a94c2 service nova] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:15.490594 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1436/2896] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:36:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:36:17.848151 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1f7c06-e9a6-4187-881f-219c3b9a0b31 service nova] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:17.903109 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1f7c06-e9a6-4187-881f-219c3b9a0b31 service nova] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:17.908929 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2eac7-0c7c-4844-9063-565ef5ea03f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a699988f08a4001b3b7d764f01f4fc5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:17.909619 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2eac7-0c7c-4844-9063-565ef5ea03f5 service nova] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:17.909899 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2eac7-0c7c-4844-9063-565ef5ea03f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:17.910172 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2eac7-0c7c-4844-9063-565ef5ea03f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:17.912247 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-72a2eac7-0c7c-4844-9063-565ef5ea03f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:17.927317 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a2eac7-0c7c-4844-9063-565ef5ea03f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:17.928973 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a2eac7-0c7c-4844-9063-565ef5ea03f5 service nova] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:17.943575 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1461/2897] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:36:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:36:22.333084 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18c2ba9-f013-4d10-b4e8-d155856a94c2 service nova] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:22.334505 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-c18c2ba9-f013-4d10-b4e8-d155856a94c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:22.375831 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18c2ba9-f013-4d10-b4e8-d155856a94c2 service nova] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:22.381642 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edcc682-e05e-49d8-b487-06779b94bf6f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30963f2f464e4f54a8aea0175370a235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:22.382289 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edcc682-e05e-49d8-b487-06779b94bf6f service placement] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:22.382569 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edcc682-e05e-49d8-b487-06779b94bf6f service placement] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:22.382846 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edcc682-e05e-49d8-b487-06779b94bf6f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:22.386671 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-6edcc682-e05e-49d8-b487-06779b94bf6f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:22.401881 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edcc682-e05e-49d8-b487-06779b94bf6f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:22.404126 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edcc682-e05e-49d8-b487-06779b94bf6f service placement] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:22.415728 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1437/2898] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:36:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:36:25.556387 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2eac7-0c7c-4844-9063-565ef5ea03f5 service nova] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:25.569310 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2eac7-0c7c-4844-9063-565ef5ea03f5 service nova] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:25.575978 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a9a0c-fb7f-401d-a039-fe778b815602 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30963f2f464e4f54a8aea0175370a235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:25.576886 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a9a0c-fb7f-401d-a039-fe778b815602 service placement] REQUEST_METHOD: `GET` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:25.577368 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a9a0c-fb7f-401d-a039-fe778b815602 service placement] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:25.577820 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a9a0c-fb7f-401d-a039-fe778b815602 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:25.579892 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-bc7a9a0c-fb7f-401d-a039-fe778b815602 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:25.598566 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a9a0c-fb7f-401d-a039-fe778b815602 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:25.600217 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a9a0c-fb7f-401d-a039-fe778b815602 service placement] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:25.616802 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1462/2899] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:36:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 03:36:25.628693 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edcc682-e05e-49d8-b487-06779b94bf6f service placement] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:25.630456 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-6edcc682-e05e-49d8-b487-06779b94bf6f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:25.736279 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edcc682-e05e-49d8-b487-06779b94bf6f service placement] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:25.741893 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9d346-432a-4283-9d2e-f5560dfe181c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:25.742774 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9d346-432a-4283-9d2e-f5560dfe181c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:25.743049 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9d346-432a-4283-9d2e-f5560dfe181c admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:25.743320 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9d346-432a-4283-9d2e-f5560dfe181c admin admin] PATH_INFO: `/v3/users/c338d0e7dce9404597cd0061e408091d` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:25.744622 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9d346-432a-4283-9d2e-f5560dfe181c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c338d0e7dce9404597cd0061e408091d)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:25.746738 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9d346-432a-4283-9d2e-f5560dfe181c admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:25.792592 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:25.805505 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-e8a9d346-432a-4283-9d2e-f5560dfe181c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c338d0e7dce9404597cd0061e408091d'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:25.817102 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-e8a9d346-432a-4283-9d2e-f5560dfe181c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c338d0e7dce9404597cd0061e408091d'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:25.826095 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-e8a9d346-432a-4283-9d2e-f5560dfe181c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c338d0e7dce9404597cd0061e408091d'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:25.832945 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-e8a9d346-432a-4283-9d2e-f5560dfe181c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c338d0e7dce9404597cd0061e408091d'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:25.833641 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-e8a9d346-432a-4283-9d2e-f5560dfe181c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c338d0e7dce9404597cd0061e408091d'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:25.839516 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1438/2900] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:36:25 2022] DELETE /identity/v3/users/c338d0e7dce9404597cd0061e408091d => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:26.024980 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a9a0c-fb7f-401d-a039-fe778b815602 service placement] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:26.037063 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a9a0c-fb7f-401d-a039-fe778b815602 service placement] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:26.104478 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c0f92-f788-4079-9e5f-75117dcff116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:26.105308 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c0f92-f788-4079-9e5f-75117dcff116 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:26.105584 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c0f92-f788-4079-9e5f-75117dcff116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:26.105853 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c0f92-f788-4079-9e5f-75117dcff116 admin admin] PATH_INFO: `/v3/projects/2bab42aa679b4182887da08409099827` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:26.107682 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c0f92-f788-4079-9e5f-75117dcff116 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bab42aa679b4182887da08409099827)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:26.109662 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c0f92-f788-4079-9e5f-75117dcff116 admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:26.161404 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-e01c0f92-f788-4079-9e5f-75117dcff116 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bab42aa679b4182887da08409099827', 'request_id': 'req-e01c0f92-f788-4079-9e5f-75117dcff116'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:26.167545 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-e01c0f92-f788-4079-9e5f-75117dcff116 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bab42aa679b4182887da08409099827', 'request_id': 'req-e01c0f92-f788-4079-9e5f-75117dcff116'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:26.177019 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-e01c0f92-f788-4079-9e5f-75117dcff116 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bab42aa679b4182887da08409099827', 'request_id': 'req-e01c0f92-f788-4079-9e5f-75117dcff116'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:26.188462 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-e01c0f92-f788-4079-9e5f-75117dcff116 admin admin] The token cache is being invalidate because project 2bab42aa679b4182887da08409099827 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:36:26.188741 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-e01c0f92-f788-4079-9e5f-75117dcff116 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:26.189360 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-e01c0f92-f788-4079-9e5f-75117dcff116 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:26.190914 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1463/2901] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:36:26 2022] DELETE /identity/v3/projects/2bab42aa679b4182887da08409099827 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:26.195203 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9d346-432a-4283-9d2e-f5560dfe181c admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:26.196659 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-e8a9d346-432a-4283-9d2e-f5560dfe181c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:26.210177 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9d346-432a-4283-9d2e-f5560dfe181c admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:26.269220 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67489113-cbd4-49f3-8d6e-b73b0127f594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:26.270111 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67489113-cbd4-49f3-8d6e-b73b0127f594 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:26.270388 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67489113-cbd4-49f3-8d6e-b73b0127f594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:26.270659 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67489113-cbd4-49f3-8d6e-b73b0127f594 admin admin] PATH_INFO: `/v3/users/539278709f6c4d8fb5bb744610531447` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:26.271886 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67489113-cbd4-49f3-8d6e-b73b0127f594 admin admin] RBAC: Authorizing `identity:delete_user(user_id=539278709f6c4d8fb5bb744610531447)` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:26.273966 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67489113-cbd4-49f3-8d6e-b73b0127f594 admin admin] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:26.325303 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-67489113-cbd4-49f3-8d6e-b73b0127f594 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '539278709f6c4d8fb5bb744610531447'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:26.337162 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-67489113-cbd4-49f3-8d6e-b73b0127f594 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '539278709f6c4d8fb5bb744610531447'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:26.346143 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-67489113-cbd4-49f3-8d6e-b73b0127f594 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '539278709f6c4d8fb5bb744610531447'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:26.353388 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-67489113-cbd4-49f3-8d6e-b73b0127f594 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '539278709f6c4d8fb5bb744610531447'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:26.354102 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.notifications [None req-67489113-cbd4-49f3-8d6e-b73b0127f594 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '539278709f6c4d8fb5bb744610531447'} {{(pid=80410) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:26.360676 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1439/2902] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 03:36:26 2022] DELETE /identity/v3/users/539278709f6c4d8fb5bb744610531447 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:26.547834 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c0f92-f788-4079-9e5f-75117dcff116 admin admin] Authenticating user token {{(pid=80409) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:36:26.549325 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.fernet_utils [None req-e01c0f92-f788-4079-9e5f-75117dcff116 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80409) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:36:26.562659 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c0f92-f788-4079-9e5f-75117dcff116 admin admin] Validating token access rules against request {{(pid=80409) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:36:26.587764 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:26.622106 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84bee43-8d3a-4bd6-9491-b3be07998e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a96d7acf93e46c7ac5700560db36dbe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73644351894d46fdb8889fca96b4c614', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80409) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:36:26.623154 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84bee43-8d3a-4bd6-9491-b3be07998e0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:36:26.623418 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84bee43-8d3a-4bd6-9491-b3be07998e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:36:26.623713 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84bee43-8d3a-4bd6-9491-b3be07998e0a admin admin] PATH_INFO: `/v3/projects/407905b0b8764df0a86e1ac9f93b682c` {{(pid=80409) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:36:26.624873 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84bee43-8d3a-4bd6-9491-b3be07998e0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=407905b0b8764df0a86e1ac9f93b682c)` {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:36:26.626844 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84bee43-8d3a-4bd6-9491-b3be07998e0a admin admin] RBAC: Authorization granted {{(pid=80409) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:36:26.682926 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-e84bee43-8d3a-4bd6-9491-b3be07998e0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '407905b0b8764df0a86e1ac9f93b682c', 'request_id': 'req-e84bee43-8d3a-4bd6-9491-b3be07998e0a'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:26.689298 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-e84bee43-8d3a-4bd6-9491-b3be07998e0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '407905b0b8764df0a86e1ac9f93b682c', 'request_id': 'req-e84bee43-8d3a-4bd6-9491-b3be07998e0a'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:26.699379 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-e84bee43-8d3a-4bd6-9491-b3be07998e0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '407905b0b8764df0a86e1ac9f93b682c', 'request_id': 'req-e84bee43-8d3a-4bd6-9491-b3be07998e0a'} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:26.710093 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-e84bee43-8d3a-4bd6-9491-b3be07998e0a admin admin] The token cache is being invalidate because project 407905b0b8764df0a86e1ac9f93b682c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80409) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 03:36:26.710373 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-e84bee43-8d3a-4bd6-9491-b3be07998e0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:26.711025 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG keystone.notifications [None req-e84bee43-8d3a-4bd6-9491-b3be07998e0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80409) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 03:36:26.712593 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: [pid: 80409|app: 0|req: 1464/2903] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 03:36:26 2022] DELETE /identity/v3/projects/407905b0b8764df0a86e1ac9f93b682c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 03:36:36.357201 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=70,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:36:36.707150 dv-cinder-828806-4-iscsi devstack@keystone.service[80409]: DEBUG dbcounter [-] [80409] Writing DB stats keystone:SELECT=39,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80409) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 03:37:20.071162 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67489113-cbd4-49f3-8d6e-b73b0127f594 admin admin] Authenticating user token {{(pid=80410) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 03:37:20.072874 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-67489113-cbd4-49f3-8d6e-b73b0127f594 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:20.150107 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67489113-cbd4-49f3-8d6e-b73b0127f594 admin admin] Validating token access rules against request {{(pid=80410) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 03:37:20.232259 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acc8b4b-4df5-4223-994f-feb95ca3d290 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '30963f2f464e4f54a8aea0175370a235', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f8398941252346468483ea0c863122f4', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80410) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 03:37:20.233025 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc8b4b-4df5-4223-994f-feb95ca3d290 service placement] REQUEST_METHOD: `GET` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 03:37:20.233296 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc8b4b-4df5-4223-994f-feb95ca3d290 service placement] SCRIPT_NAME: `/identity` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 03:37:20.233564 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acc8b4b-4df5-4223-994f-feb95ca3d290 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80410) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 03:37:20.235607 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.fernet_utils [None req-6acc8b4b-4df5-4223-994f-feb95ca3d290 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80410) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 03:37:20.247924 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acc8b4b-4df5-4223-994f-feb95ca3d290 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 03:37:20.249570 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acc8b4b-4df5-4223-994f-feb95ca3d290 service placement] RBAC: Authorization granted {{(pid=80410) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 03:37:20.260458 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: [pid: 80410|app: 0|req: 1440/2904] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 03:37:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 03:37:30.255438 dv-cinder-828806-4-iscsi devstack@keystone.service[80410]: DEBUG dbcounter [-] [80410] Writing DB stats keystone:SELECT=58 {{(pid=80410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}